Ableton and Ableton Push Hacking

For those who have been tracking this blog, it's been obvious that I've recently been spending more and more time with the Ableton Push.

If you don't know what this is please see the following...
https://www.ableton.com/en/help/learn-push/
http://blog.dubspot.com/learned-30-days-ableton-push/
http://sonicbloom.net/en/two-weekends-with-ableton-push-part-1-review/
http://emilehoogenhout.com/reviews/ableton-push-an-exploration-emile-hoogenhout/
https://www.youtube.com/results?search_query=Ableton+Live+9+%26+Push+Masterclass+Webinar  
Easy to miss Push features
https://forum.ableton.com/viewtopic.php?f=55&t=196560
Helpful Push information.
https://forum.ableton.com/viewtopic.php?f=55&t=215744

Jeremy Ellis meets Ableton Push
https://www.youtube.com/watch?v=EdCgA54g-aM
Mad Zach Push Performance Walkthrough
https://www.youtube.com/watch?v=iSZm4ECbE5k
Decap Push Performance
https://www.youtube.com/watch?v=ZOtj5WtkR1Q

It's basically an advanced, modern musical instrument/MIDI controller.

There have been others who have attempted to de-compile and extend/modify the behaviour of the device but while information and the extensions that have been provided have been interesting and useful they have been somewhat limited.
https://github.com/gluon/AbletonLive9_RemoteScripts
http://julienbayle.net/ableton-live-9-midi-remote-scripts/
http://julienbayle.net/ableton-push/
ableton: just release the py midi remote scripts
https://forum.ableton.com/viewtopic.php?f=1&t=134338
Live 9 MIDI Remote Scripts revealed...
https://forum.ableton.com/viewtopic.php?f=1&t=190069
http://livecontrol.q3f.org/ableton-liveapi/articles/introduction-to-the-framework-classes/

I'm beginning to understand why. The following link provides an update to automatically generated documentation (via epydoc) of decompiled Ableton Remote Script code (my scripts for decompilation and automated documentation have been included in the package).

https://sites.google.com/site/dtbnguyen/Decompiled-Documentation-Ableton-MIDI-Remote-Scripts.zip

https://wiki.python.org/moin/DocumentationTools
http://stackoverflow.com/questions/1125970/python-documentation-generator
http://sphinx-doc.org/
https://packages.debian.org/search?keywords=python-sphinx
https://en.wikipedia.org/wiki/Comparison_of_documentation_generators
https://packages.debian.org/stable/python/python-epydoc
https://packages.debian.org/wheezy/python-epydoc

If you want to make any additional modifications of behaviour you'll need to be aware of the following:
- you'll need to catch up on your Python coding
http://stackoverflow.com/questions/36901/what-does-double-star-and-star-do-for-python-parameters
http://stackoverflow.com/questions/400739/what-does-asterisk-mean-in-python
https://en.wikipedia.org/wiki/Pickle_%28Python%29
http://stackoverflow.com/questions/514371/whats-the-bad-magic-number-error
http://stackoverflow.com/questions/12233837/bad-magic-number-while-trying-to-import-pyc-module
http://stackoverflow.com/questions/6034621/bad-magic-number-error-persists-even-after-rebuilding-the-pyc-file
https://shankaraman.wordpress.com/tag/how-to-fix-runtimeerror-bad-magic-number-in-pyc-file/
http://www.tutorialspoint.com/python/python_exceptions.htm
http://www.linuxtopia.org/online_books/programming_books/python_programming/python_ch17s03.html
- you'll need knowledge of how the device works, music and mathematical theory, Ableton, and core computing knowledge. It is not sufficient to know how they work seperately. You need to know how everything fits together.
- sounds obvious but start small and move up. This is critical particularly with reference to the awkward style of programming that they can sometimes resort to. More on this to below
- the code can vary in quality and style quite significantly at times. At times it seems incredibly clean, elegant, and well documented. At other times, it there is no documentation at all and doesn't seem to be well designed or engineered or have keep maintenance in mind. For instance, a commonly used design pattern is MVC. This doesn't seem to follow that. They use a heap of sentinels throughout there code. Moreover, the characters that are used can be a bit confusing. They don't use preprocesser directives/constants where they may be better suited. If you break certain aspects of the code you can end up breaking a whole lot of other parts. This may be deliberate (to reduce the chances of third party modification which is likely particularly as there seems to be some authentication/handshake mechasnisms in the code to stop it from working with 'uncertified devices') or not (they lack resources or just have difficult timelines to deal with)
https://mail.python.org/pipermail/tutor/2003-July/024206.html
http://stackoverflow.com/questions/2682745/creating-constant-in-python
- be prepared to read through a lot of code just to understand/make a change to something very small. As stated previously strictly speaking at times they don't adhere to good practice. That said, other aspects can be changed extremely easily without breaking other components
- due to the previous two poits it should seem obvious that it can be very difficult to debug things sometimes. Here's the other thing you should note,
- the reason why Ableton suffers from strange crashes and hangs from time to time becomes much more obvious when you look at the way they code. In the past, I've built programs (ones which rely on automated code generation in particular) that relied on a lot of consecutive steps that required proper completion/sequencing for things to work properly. When things work well, things are great. When things break, you feel and look incredibly silly
- you may need to figure out a structure for ensuring and maintaining a clean coding environment. I try to have two screens with one for clean code and another for modified code. Be prepared to restart from scratch by reverting to a clean pyc code and only one or a small number of modified py files.
- caching occurs in situations where you may not entirely expect. If you can not explain what is happening and suspect caching just restart the system. Better yet, maintain your development environment in a virtual machine to reduce hardware stress caused by continual restarts.
- you will need patience. As stated previously, due to the way code has been structured (sometimes) you'll need to understand it properly to allow you to make changes without breaking other parts. Be prepared to modify, delete, or add code just to help you understand it
- if you've ever dealt with firmware or embedded devices on a regular basis you would be entirely familiar with some of what I'm talking about. Like a lot of embedded devices you'll have limited feedback if something goes wrong and you'll be scratching your head with regards to how to work the problem
http://dtbnguyen.blogspot.com.au/2012/07/if-only-reading-were-easier.html
http://dtbnguyen.blogspot.com.au/2012/08/funky-firmware.html
You may require a lot of Linux/UNIX based tools and other debugging utilities such as IDA Pro, Process Explorer and Process Monitor from the Sysinternals Suite. Once you examine Ableton using such utilities, it becomes much clearer how the program has been structured, engineered, and designed. One thing that can cause mayhem in particular is the Ableton Indexer which when it kicks in at the wrong time can make it feel as though the entire system has frozen.
https://technet.microsoft.com/en-us/sysinternals/bb545021.aspx
https://en.wikipedia.org/wiki/Sysinternals
Ablton indexing Crashes
https://forum.ableton.com/viewtopic.php?f=1&t=206205
(42474187) Disable "Ableton Index" possible?
https://forum.ableton.com/viewtopic.php?f=52&t=191573
http://audiosex.pro/index.php?/topic/11798-ableton-9-killing-index-process-to-speed-up-workflow/
http://audiosex.pro/index.php?/topic/11815-a-little-app-to-speed-up-live-9-mac-only/
https://www.ableton.com/en/help/article/indexer-crash/
The actual index file/s are located at
C:\Users\[username]\AppData\Roaming\Ableton\Live 9.1.7\Database
- the most relevant log file is located at,
C:\Users\[username]\AppData\Roaming\Ableton\Live 9.1.7\Preferences\Log.txt
The timestamps works on the basis on amount of time since program startup. Time of startup is clearly outlined.
Ableton takes 130 - 2 mins to start up ?
https://forum.ableton.com/viewtopic.php?f=1&t=6308
Delete it if you need to if you get confused about how it works.
https://www.ableton.com/en/help/article/live-crash-packs/
https://www.ableton.com/en/help/article/live-crashing-what-should-i-do-now/
https://www.ableton.com/en/help/article/how-to-out-of-memory/
- be aware that there are some things that you can't do anything about. The original Novation Launchpad was considered somewhat sluggish in terms of refresh rate and latency. The electronics were subsequently updated in the Novation Launchpad S to deal with it. You may encounter similar circumstances here.
Push browser - slow, freezing, sluggish :(
https://forum.ableton.com/viewtopic.php?f=55&t=215539
- they have a strong utility/systems engineering mentality. A lot of files are archives which include relatively unobfuscated content. For instance, look in a
C:\Users\[username]\AppData\Roaming\Ableton\Live 9.1.7\Live Reports
and you'll find a lot of 'alp' files. These are 'Crash Reports' which are sent to Ableton to help debug problems. Rename them to a gz file extension and run it through hexedit. Same with 'adg' audio device group files. Rename to gz and gunzip to see a flat XML file containing some encoded information but mostly free/human readable content. It will be interesting to see how much of this can be manually altered achieving flexibility in programming without having to understand the underlying file format.
https://www.ableton.com/en/help/article/filetypes-used-by-ableton/
- each version of Ableton seems to have a small version of Python included. To make certain, advanced extensions others have suggested installing a libraries seperately or a different version of Python...
- be prepared to learn multiple protocols and languages in order to make the changes that you want
How to control the Push LCD text with sysex messages
https://forum.ableton.com/viewtopic.php?f=55&t=193744

http://www.snoize.com/MIDIMonitor/

http://archive.monome.org/community/discussion/1648/share-your-arduinomonome-code/p1
http://hackaday.com/2008/08/13/rgb-monome-clone/
https://en.wikipedia.org/wiki/Protofuse
http://julienbayle.net/works/creation/protodeck-midi-controller-for-ableton-live/

For a lot of people, the device seems incredibly expensive for what amounts to a MIDI controller. It was much the same with me. The difference is that it's becoming increasingly clear how flexible the device can be with adequate knowledge of the platform.
http://www.nativekontrol.com/
http://blog.dubspot.com/5-ableton-push-software-devices/

Push feature requests
https://forum.ableton.com/viewtopic.php?f=55&t=192110
The Ableton Push is a good platform but it will never realise it's full potential if the software isn't upgraded.

If you are interested in signing up to test the latest Beta version of Ableton please see the following...
https://ableton.centercode.com/signup/

- as usual thanks to all of the individuals and groups who purchase and use my goods and services
http://sites.google.com/site/dtbnguyen/
http://dtbnguyen.blogspot.com.au/