Building the Magus firmware

OWL Magus
, ,
#121

I feel like I just made a LOT of posts so here is a recap for @mars of all my tests.

v20.7: Good: USB DEVICE works with Mac and Windows computers. Bad: MIDI devices plugged into USB HOST drop events.

10a981bf68a2: Good: USB DEVICE works with Mac and Windows computers. Bad: MIDI devices plugged into USB HOST drop events. (Same as v20.7)

ddfe79cf55f1 merged with a447765258d9: Good: USB DEVICE works with Mac and Windows computers. Bad: MIDI devices plugged into USB HOST drop events. (Same as v20.7 and 10a981bf68a2.)

732ceac17cb merged with a447765258d9: Good: USB DEVICE works with Macintosh. USB HOST works flawlessly. Bad: UDB DEVICE does not work with Windows.

efaaa7661bde: Bad: USB DEVICE does not work with Windows computer. Did not test with Macintosh. USB HOST does not work at all (does not even supply power).

d6c58cf8ec6126: Good: USB DEVICE works with Macintosh. Bad: USB DEVICE does not work with Windows computer. USB HOST does not work at all (does not even supply power). (Same as efaaa7661bde.)

Encoder knobs work correctly on all above commits. No commits are having problems with power or power draw. I sent an MS employee a log of efaaa7661bde failing to connect to Windows and he said this. I am running my Magus now with 732ceac17cb+a447765258d9 and I consider the device fully “usable” in this state.

1 Like
#122

Regarding the “patches disappearing” issue. The problem is that patch strorage size was reduced from 640kb to 512kb. This makes firmware storage start from another address and old patches “disappear”

@mars, I can tell for sure that this can lead to a corrupt storage and bricked device with current firmware code (it can interpret junk data as invalid memory addresses when validating storage blocks and this is leading to hard fault). I suppose you changed it after running out out of flash for debug builds. It’s better to just compile with -O in such case. Or I’ve got another proposal - on Daisy I had some code to move pow/log tables to resources instead of FW itself. I could make a PR that enables that in debug builds and add sysex files for those tables.

#123

It seems like most people will only be upgrading one way, so it might be reasonable to erase the firmware storage on upgrade rather than just moving the start pointer?

Magus patch projects log
#124

Thank you @mcc for this rundown, it helped me try to pinpoint the Windows driver issue/s.

For a long while I thought it was due to library changes in the STM32Cube_FW_F4_V1.25.1 updates. But no, the libs check out alright.

Turns out Windows doesn’t like audio devices with zero channels. And that there’s some issue with the new audio driver, so that compiling it in MIDI-only mode caused it to hang whenever there was USB activity. And that for some reason Windows doesn’t like our driver with output channels configured.

And most annoying of all, because I knew about it but still didn’t realise that’s why my testing failed: Windows has issues with MIDI devices resetting (as when you’ve just flashed it) while there’s a program connected to it (such as Chrome or Edge). Mostly it works, but sometimes it hangs the browser and does all sorts of weird things. Frequently it looks as if the device is connected, but no data is sent or received. Restarting Chrome doesn’t help. Sometimes doing a ‘hard’ restart works, ie going to Task Manager and killing all the background Chrome processes. Suffice to say, Windows and I don’t enjoy each other’s company.

Long story short: Magus firmware, develop branch should now work on Windows.

Key configuration settings:

#define AUDIO_CHANNELS              2
#define USE_USBD_AUDIO
#define USE_USBD_AUDIO_TX  // microphone
/* #define USE_USBD_AUDIO_RX // speaker */

Upshot is, you’ve now got a USB Audio interface that gives you the Magus stereo inputs as 48kHz 16-bit audio.

2 Likes
#125

Interesting. I’ve seen occasional freezes on Daisy after boot for first few seconds with USB audio enabled. I thought this could be semihosting printf slowing things down, don’t remember if this issue was present in release builds too.

#126

Hi !

I’ve been reading this discussion a few times as I’m trying to update my magus firmware for the first time over linux.
You’ve been through many things and I probably didn’t get everything so please correct me if I’m wrong. I will try to synthetize what’s been done.
So on one side you’re talking about general firmware update and on another side how to custom your own firmware. You managed to correct encoders, usb host perf (midi?) and in the latest release (v20.10) you’ve opened audio over usb. Does that make magus a DC coupled audio interface ?

I understood you first need to build the firmware and then flash with midi sys message over usb-B. I managed to build but I don’t understand how to flash.

What I did:
clone v20-10 branch
git submodule init
git submodule update
TOOLROOT ?= /home/me/bin/gcc-arm-non-eabi-9-2020-q2-update/bin directory in common.mk
cd /home/me/Document/RebelTechnology/OpenWare
make clean
make magus (this builds the Magus.bin in the Build folder)
clone FirmwareSender
make (in FirmwareSender linux folder)

I reckon I still need to generate Magus.sys from Magus.bin and then send it to Magus via USB-B.
So a few questions:
Do I still need STMcubeMX to generate files from Magus.ioc ? If yes, when do I have to do this ?
How and when to use FirmwareSender ?
Do I need any other connection than USB-B ? Such as hardware programmer ?
Stupid question: Does Magus have to be recognized as a USB device when updating firmware ?

@antisvin In your first message, you mentioned a few command line:

I’m sure there are elements I need here but I just don’t understand the firmwaresender line. Is it using FirmwareSender.jucer ?

#127

Hi @AlWared

FirmwareSender is the program that can convert binary firmware image to MIDI SySex file (Magus.syx in this case) that you can load. So you need it if you’re building firmare. When loading official releases you can just download SySex file and send (play) to the OWL MIDI device. This is the project repository - GitHub - pingdynasty/FirmwareSender: Utility program to package and send firmware and dynamic prog

I think that right now you shouldn’t need to run CubeMX just to build firmware. At least that was the intention of recent updates to project infrastructure. So when you run “git submodule init/update” commands, you should get all necessary libraries to build FW. CubeMX should only be needed for new devices or if you’re making some kind of hardware modifications, etc.

But if you want to use USB Audio, you’ll have to build from develop branch - it’s not released yet, but I think Martin will be making a release very soon. It will be the version 21.0. Currently there’s one known problem with it - new USB MIDI host code stops working if you disconnect and reconnect a device to it. This requires a reboot if you want to switch controllers, so not a huge issue. V20.10 was a release made for the new Lich module, that’s why there was no release made for other devices. It doesn’t have USB audio yet.

Currently Magus configs enables capturing from its audio inputs - so yes, it can be used as DC coupled audio interface. MIDI still works on it at the same time. I didn’t spend much time testing them, but it seems pretty stable to me. I think Martin did an excellent job here, because USB code is quite complicated.

You don’t need to use programmer just to load firmware from a SySex file. But if you will load firmware that doesn’t work on Magus, you will need a programmer to restore firmware that works. There’s a newer version of bootloader that can detect this situation, so if firmware doesn’t boots you’ll be back to bootloader mode. However, you also need a programmer to update bootloader - there’s no bootloader update procedure on device itself. That is why I would suggest to get programmer if you want to install any experimental code (modified firmware or development branch).

#128

AlWared, does this document help at all?

It gives instructions for Windows but I have “if you’re on Linux”… branches at some point.

The document describes the various cases of installing an official firmware, installing a self-built firmware, installing the bootloader (needed for custom firmwares), and running STMCube/building the IOC (not needed anymore)

2 Likes
#129

Thanks for your help !
I think I misunderstood with the usb audio. So now the master branch is at release 20.7
So I cloned openware. I git init and git update. Then I change the common.mk file.
And then when I try make magus I get this FreeRTOS.h missing file

./Src/freertos.c:52:10: fatal error: FreeRTOS.h: No such file or directory
52 | #include “FreeRTOS.h”
| ^~~~~~~~~~~~

Do I even need to run this make magus ? Where do I find the syx file ?

#130

Sysex releases are available here: https://github.com/pingdynasty/OpenWare/releases/tag/release
You only need to build from source if you are modifying the firmware, or want to be on the bleeding edge of development (there could be some pain involved).

If so, do git submodule init (once) and git submodule update (if you change branch) to get the right version of the libraries.

#131

Got two more Magus questions…

  1. I suddenly discovered firmware sender can no longer save patches with my currently installed firmware. -run works but -store does not. I’m trying to send with:

     (cd .. && make patch TOOLROOT= PATCHNAME=Midi2CVTriplet PLATFORM=Magus && ../FirmwareSender/Builds/MacOSX/build/Debug/FirmwareSender -in Build/patch.bin -out OWL-MIDI -store 2)

    I’m running 732ceac17cb merged with a447765258d9 on the device. I’m using d9e0d958b3fb of FirmwareSender

    Other oddities: Can’t seem to upload from rebeltech/patch-library either now, frequently when I interact “INVALID SYSEX” prints on one side or the other. I at one point recently was able to upload but they only seemed to upload with the name “Template”.

    Any idea what I could do here? Is USB HOST working with master head right now?

  2. I got a module called a “Chainsaw” that’s 3-voice oscillator with individual CV control. I made a Magus patch to convert MIDI into 3-channel paraphonic CV specially for use with this. Since there are only 2 audio-style outputs, I set the 3 voices on parameter outputs. The parameter outputs appear to use 0.0 for 5V and 1.0 for 10V. Testing with this patch, the 3 voice inputs sound “wrong”. It is hard to describe, they just don’t seem to be harmonizing correct. The problem here could be at the Magus end or could be a problem with the Chainsaw. A couple of questions I’ve got: (1) is my notion of the range (0=0, 1=10V) correct? (2) Looking in the scope it seems like the parameter outputs are slightly noisier than the audio outputs. I don’t know if it’s significant enough to effect perceived pitch but putting my scope in tuner mode it seems to have more of a problem getting a lock on a a pitch. Is there anything different between the audio and parameter outs that is controllable from software? (3) There’s this whole “calibration” feature— I have not used it. Does this apply or to audio only or to the parameters also?

Noisy parameter patch points
Resources usage
#132

That error has been frequently happening on MacOS lately. Sysex error messages where disabled in latest firmware release for this reason.

USB host is working, unless you have some unknown problem. It still gives you an error if you reconnect device to it, device restart is required. I had previously ran into some problems, turns out I had a faulty cable inside my Magus. So at least I can use it now.

Calibration is for audio channels only and Magus is using 0-10V output range, correct. DAC is 12 bit vs 24 bit audio codec. As I’ve mentioned a few times, currently Magus has its UI update code locked to display update rate, which means that effectively it’s sampled at 50Hz. I would consider it one step above broken. I’ve made some updates to resolves this issue on other hardware in my forked code, might be good time to try to backport it to upstream.

#133

Yes everything should be working reliably afaik.
Best thing to do is to upgrade to v21.0 then if your problems persist I will try to reproduce.

#134

I first had issues with my ubuntu vm: sysex program errors, buffer overflow. I could neither load patches, nor upload firmware.
Then I uploaded v21.0 through chrome on windows and it’s all working properly now !
Thanks for helping

1 Like
#135

I have got around to installing tag v21.0, I compiled the sysex myself… I think it works because it says “Magus v21.0” on the screen. Oddly I am noticing now that I’ve uploaded, I forgot to turn on the watchdog before building, but it’s working anyway (and not randomly rebooting) for some reason.

But, unfortunately, this very strange error where I can no longer store anything onto the device is persisting :frowning: If I run ../FirmwareSender/Builds/MacOSX/build/Debug/FirmwareSender -in Build/patch.bin -out OWL-MIDI -store 6 it is ignored, if I use the “store” button on the Rebeltech “patches” page it does not work. I have five patches installed and I can select any of them fine but it seems to ignore any attempt to upload a new patch.

What could possibly be wrong? Any idea how to regain the ability to save patches?

I did the above tests on my mac. I notice that although when I plug in to my Windows machine it says “setting up OWL-MIDI”, the Rebeltech “patches” page is unable to connect to the magus on the Windows machine. This makes me worry the v21.0 did not actually upload, somehow, if USB DEVICE is expected to work with Windows now. The firmware uploader page said “Firmware upload complete” though.

#136

IIRC, watchdog doesn’t reboot randomly, it would do that after 8 seconds it gets enabled and not reset. Initially it was turned on in newer version of bootloader, but now we do that in FW too. So it’s safe to always enable it in firmware by default, which was done a while ago. You still need to have an updated bootloader in case if something totally broken is flashed, because obviously you’ll get reset to bootloader only if WD was enabled before it happens.

Also, I’ve made a bootloader update procedure that allows flashing it by MIDI - no programmer required. That includes toggling flash lock options. But we don’t have a web UI for that yet.

So if you’ve seen random lockups when device was running, I would expect that to mean that your patch was hanging and you were triggering watchdog reset.

Regarding patch storing problems, we have a new command for loading patches by name rather than by slot number. FW code for storing patches had some changes, but the new command is not used and I think that the storage should not require formatting just because of this. You may still try that in case if nothing else helps.

One obvious problem in your post is that you’re using OWL-MIDI as device name, but now each device its own name in FW, so it should be -out OWL-MAGUS. Specifically:

  • OWL-MIDI used by old booltoader and firmware
  • OWL-$DEVICE is unique for each device, OWL-MAGUS in this case
  • OWL-BOOT is used by newer bootloader builds
#137

Okay, so I just need to update FirmwareSender to load a patch by name…?

#138

Actually we have a new command for storing resources by name, not patches. Things like samples. Patches are still stored under a slot number, so that they can be sequentially accessed.

#139

My bad, haven’t realized that it doesn’t apply to patches (since they were handled the same way as resources before). But either way, old version of FirmwareSender still works, I haven’t rebuild mine for ages. I’ve only mentioned this as one of candidates for breaking something.

It’s kind of hard to understand what exactly is wrong with that FW update. The most reasonable approach would be to erase patch storage and flash official firmware release. Then you’ll have a firmware version that is expected to work and no possibly corrupt storage.

#140

v21 did not break patch storage, it broke sometime before that when I was rapidly reflashing custom firmwares.

How do I erase patch storage? I have access to the web pages and “FirmwareSender”.