The manual Arduino Mac setup works just fine, assuming you do it properly. The only problem is that you're basically locked into using that Arduino install for flashing the firmware, and nothing else. The good news is that you didn't break it-- you can upload the firmware with USBTinyISP, or if you're certain the bootloader is installed properly, use the standard AVRISP MkII approach.
You absolutely have to use Arduino 1.8.5, though. I'm pretty sure it has something to do with the specific Arduino and Sanguino cores used for the board.
I wound up putting Arduino 1.8.5 on the Raspbian install that I put OctoPi on. If I need to update something in the firmware, I just VNC into my Pi and do it there. I have a lot of different Arduino projects that I mess around with, so I didn't want to limit either my MacBook or my Linux workstations Arduino installs to just having those cores installed.