Mouse Movement XY-Pad to Midi

Discussion in 'PC' started by kostovas, Aug 12, 2018.

  1. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    Hello all

    It is finally time for me to contribute.

    I am finally able to use my computer mouse to control any MIDI CC I want and wanted to share this with you.
    You will need GlovePIe(which is a really old gem) and loopMidi or any other virtual MIDI software.

    Once you run GlovePie you can put the following code in:
    Code:
    Pie.MouseDPI = 800            //Set mouse DPI
    
    If Key.Tab then               //Check if a key is pressed, in this case Tab
    
         mouse.Swallow = true          //We disengage the cursor
         midi3.channel1.Expression =  EnsureMapRange(mouse.DirectInputX, -2000, 2000, 0, 1)
                                  //Get the direct input(X-axis) from mouse, (-2000, 2000) original range in Mickeys, converted range (0, 1)
         midi3.channel1.Breath = EnsureMapRange(mouse.DirectInputY, 2000, -2000, 0, 1)
                                  //Same for Y-axis
    Else
    
         mouse.Swallow = false         //Release the cursor to Windows
    
    End if
    
    //EnsureMapRange makes sure the output does not go above the set limits
    The mouse movement turns into MIDI while you hold the Tab key.

    by using the GUI tab in GlovePie you can see the available midi devices and their numbers.
    It is also possible to use multiple mice at the same time for different MIDI cc's, use the mousewheel or any Button on the mouse and any button from the Keyboard for MIDI. I also believe you can use multiple keyboards in the same way. You need to put a number next to mouse.
    Code:
    mouseX.DirectInputX
    Just replace the first X with a number. it might take some time to finder the number.

    I recommend using the free MID-OX for debugging so you know what is working before opening your DAW.

    I hope you find this guide useful.

    Enjoy.

    PS 1. If you need any help don't hesitate to ask or if you have any ideas and suggestion, the possibilities are almost endless.
    PS.2. The download source are the ones I used because the original websites did not work.
    Ps 2. I will try to convert this into a FreePIE script which is currently maintained.
     
    • Like Like x 4
    • Interesting Interesting x 2
    • Useful Useful x 2
    • Love it! Love it! x 1
    • List
  2.  
  3. metaller

    metaller Audiosexual

    Joined:
    May 28, 2016
    Messages:
    760
    Likes Received:
    524
    Location:
    Persia
    Thanks for the tutorial. :wink:
    Do you have any experience with TouchOsC and Android phones? producing midi using Gyro sensors of the phone?
    I am looking for this program.
     
  4. kooper

    kooper Platinum Record

    Joined:
    Jun 6, 2011
    Messages:
    563
    Likes Received:
    173
    I am wondering about practical application.
     
  5. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    metaller thanx for your comment, I own TouchOSC but I didn't have it installed and was unaware that it can send accelerator data through OSC. It seems that GlovePIE supports OSC so I will try to figure something out from the documentation.

    @kooper I would say that the practical applications are endless from turning on your coffee machine in the morning to writing Expression/Breath/.... data to a MIDI track. Personally I can write the Expression on a track quite smoothly with this.
     
  6. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    @metaller I had a look at the OSC accelerometer but there are a few problems that can't be solved in GlovePie.
    The OSC message contains x,y,z packed in /accxyz and can't be unpacked, same goes for XY pads.
    OSC works for buttons and faders but I can only get the X value from the XYpad and the Gyro.

    I think it is better to use PureData or FreePIE for that or you can always use only the X axis.

    Hmmm, now that I think about it, it might be easier and faster to use python with pyOSC and pyMidi.
     
  7. Xupito

    Xupito Audiosexual

    Joined:
    Jan 21, 2012
    Messages:
    6,956
    Likes Received:
    3,830
    Location:
    Europe
    Very interesting. I use Bidule to use my Gamepad as MIDI controller but the possibilities of pure programming are great.
    I'll have a look at FreePIE.
     
  8. metaller

    metaller Audiosexual

    Joined:
    May 28, 2016
    Messages:
    760
    Likes Received:
    524
    Location:
    Persia
    This will be great. :headbang: I have no experience with implementing the code for this applications. If you can do it, I will really appreciate it.
    Look at this:

    https://play.google.com/store/apps/details?id=com.hollyhook.oscHook

    Also this:
    https://github.com/shengpo/processing_snips/tree/master/touchOSC_AccTest


    and also this:



    BTW, your program for mouse expression works great. :metal:
     
    Last edited: Aug 13, 2018
  9. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    @metaller Thanks man, it means a lot to me. You can change the expression to any CC really.

    Yeah I have already seen these videos, that is why I said it might be better to use PureData.

    However!!!! The android app you sent rocks man, what an absolute gem. Installed it on my phone and I can see that it splits all the information for Pitch, Roll and it has so many more. I believe I can work something out with GlovePie easily, I'll let you know later today.

    @Xupito Not really familiar with Bidule, does it convert OSC to midi?

    PS. Also need to see if I can get some kind of low pass filter for the data from the accelerometer else it will jump like crazy.
     
    Last edited: Aug 13, 2018
  10. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    Okay here it is.

    Code:
    osc.ListenPort = 8000;        //What port to listen
    osc.Listening = true;         //Start listening
    
    If Key.Tab then               //Check if a key is pressed, in this case Tab
    
    midi3.channel1.Expression =  Smooth(EnsureMapRange(osc.orientation.pitch, 80, -80, 0, 1),10, 0.01)
                                  //Smooth() is used as a low pas filter
    midi3.channel1.Breath = Smooth(EnsureMapRange(osc.orientation.roll, 100, -100, 0, 1),10,0.01)
                                  //Same for Y-axis
    
    End if
    
    //EnsureMapRange makes sure the output does not go above the set limits
    You will also need oscHook from the play store as @metaller suggested. (This means the above script only works with oscHook)
    Additionally you will need GlovePIE 0.43 as there is probably a bug in 0.45 and can't read OSC messages.

    The pitch of the phone is the Expression and the roll is Breath. I tested it but if you have any problems leave a message.
     
    • Love it! Love it! x 1
    • Useful Useful x 1
    • List
  11. metaller

    metaller Audiosexual

    Joined:
    May 28, 2016
    Messages:
    760
    Likes Received:
    524
    Location:
    Persia
    In case anyone does not know how to setup OSC connection (on windows):
    0. connect your pc and android phone to the same wifi network
    1. run cmd.exe as admin
    2. type and enter:
    Code:
    ipconfig
    3. write down the number wrote in front of ipv4 or ipv6
    4. open oschook app, and press 3 points on the top right corner of the screen, press IP/port setup
    5. enter the same IP you get in 3 step
    6. enter the same port on GlovePIE script and oschook, for example, 8000 on both, then press ok
    7. press start in oschook
    8. press run, and hold the tab on the keyboard to start getting sensor information into midi
     
    Last edited: Aug 14, 2018
    • Like Like x 1
    • Interesting Interesting x 1
    • List
  12. metaller

    metaller Audiosexual

    Joined:
    May 28, 2016
    Messages:
    760
    Likes Received:
    524
    Location:
    Persia
    Thanks for the script. :metal:
    The midi changing is a little jumpy and has delays, do have any idea to make it more smooth and accurate?

    Also, how can I assign Z axis to control sth?

    Update: decreasing OSC timing to 50ms was helpful.
     
    Last edited: Aug 14, 2018
  13. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    @metaller Yeah forgot to tell you about the timing, unfortunately oscHook goes only down to 50ms (for now) where TouchOSC I believe uses 20ms. In general the accelerometer is quite jumpy and you need to fine adjust the Smooth() function(Look at the last 2 number for that in the brackets).

    Also in your points, if it is not working check your firewall or your routers ports.

    For instance If you see at the OSC address setup in oscHook it says "/accelerometer/orientation/azimuth"
    Just type that in GlovePIE like so:
    Code:
    osc.accelerometer.orientation.azimuth
    and put in the code in the same way roll and pitch is. You can do the same for the other data like the rotation Vector.

    I believe the Z axis measures gravity so it will only change when you move the phone up or down but it will always reset at the new position, I just need to figure out how to get that information as distance.
     
  14. Xupito

    Xupito Audiosexual

    Joined:
    Jan 21, 2012
    Messages:
    6,956
    Likes Received:
    3,830
    Location:
    Europe
    It's a modular audio software similar to a Max/MSP/PureData graphical editor. It can run both standalone and VST. It has an HID input block which supports pretty much all (mouse, gamepad, keyboard,..).
    Also, it supports OSC client and server but I don't know much about using it with mobile devices. I'm convinced it can convert from OSC to midi because that's where Bidule excels, with MIDI.

    The main con is that you can't write code directly, for fine-tuning like the topic of this thread nothing like programming. For that you can use any VST2/3 plugin that supports programming, like Reaper JS. Because Bidule can host almost any VST2/3 plugin.

    I use MIDI Guitar 2 for that, it has an amazing Lua-based midi editor embedded.
     
    Last edited: Aug 14, 2018
    • Interesting Interesting x 1
    • List
  15. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    @Xupito That's interesting man, might be a better solution to what I want to achieve. Will definitely check it out. Thanks.
     
  16. Xupito

    Xupito Audiosexual

    Joined:
    Jan 21, 2012
    Messages:
    6,956
    Likes Received:
    3,830
    Location:
    Europe
    Let me know if you want some help to start with the Bidule thing. This soft deserves more recognition and guides (well, it's commercial, but even so lol)
     
  17. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    Managed to get this far with Python.



    Works really nice however to make it useful for general use it will definitely need a GUI.

    Hmmm....:cool:
     
    • Like Like x 1
    • Love it! Love it! x 1
    • List
  18. metaller

    metaller Audiosexual

    Joined:
    May 28, 2016
    Messages:
    760
    Likes Received:
    524
    Location:
    Persia
    I do not have experience helping you in creating a GUI. However, really interested to see the final product. :headbang:
     
  19. kostovas

    kostovas Kapellmeister

    Joined:
    Sep 15, 2013
    Messages:
    95
    Likes Received:
    62
    @metaller Here it is man, OSCtoMIDI.

    I believe everything is self explanatory. In the Server IP field you just put your computer IP.
    There are no warnings for user errors(like wrong inputs, letters instead of numbers...just didn't have the time) so make sure you input correct numbers :-P. It would be great if you can give me some feedback and maybe some feature requests.

    Currently it only works with TouchOSC and the roll-axis. I have the other axis as well but didn't have time to test so I have them disabled.

    There are a lot of things I want to implement but don't have the time at the moment(moving houses next week and some travelling after) but I think they will be simple to implement. Maybe I have to create a new topic for this.

    I hope it is usable and everyone can enjoy this.
     
  20. metaller

    metaller Audiosexual

    Joined:
    May 28, 2016
    Messages:
    760
    Likes Received:
    524
    Location:
    Persia
    Oh thank you very much :bleh::mates::mates::bow: Let me test it.
     
Loading...
Similar Threads - Mouse Movement Midi Forum Date
Mouse movement causing interference in monitors Soundgear Sep 28, 2015
R.I.P. Jeremiah Green, Founding Member & Drummer of Modest Mouse, Dies at 45 AudioSEX Memorial Jan 3, 2023
Logitech MX Vertical Ergonomic Mouse - Is it worth it? Computer Hardware Aug 18, 2022
Anyone have a led mouse mat? Studio Jun 9, 2022
Logitech Lift: ergonomic mouse against tendinitis Computer Hardware Apr 21, 2022
Loading...