OWL on plugdata

OWL
#12

I’ve started some work to try and bash together a minimal prototype for adding OwlProgram to the plugdata toolchain and into the plugdata compile dialog.

So far I’ve managed to successfully build and flash the test-patch: https://github.com/RebelTechnology/OwlProgram/blob/develop/TestPatches/HeavyTest/HeavyTest.pd

However using this patch from the Patch Library I am getting linker errors regarding the patch size: https://www.rebeltech.org/patch-library/patch/_Lich_Windrone#Source

/home/dreamer/Sources/_projects/_wasted/_hvcc/plugdata-heavy-toolchain-wstd/Heavy/bin/../lib/gcc/arm-none-eabi/10.2.1/../../../../arm-none-eabi/bin/ld: ..//patch.elf section `.text' will not fit in region `PATCHRAM'
/home/dreamer/Sources/_projects/_wasted/_hvcc/plugdata-heavy-toolchain-wstd/Heavy/bin/../lib/gcc/arm-none-eabi/10.2.1/../../../../arm-none-eabi/bin/ld: region `PATCHRAM' overflowed by 632400 bytes
collect2: error: ld returned 1 exit status
make[1]: *** [compile.mk:179: ..//patch.elf] Error 1
make: *** [Makefile:110: patch] Error 2

The patch size seems to be extremely large somehow, any insight into what’s going on would be helpful.

However I’m not using the regular make HEAVY=patch method of building, because we first export the OWL code directly using our PyInstaller version of HVCC in the plugdata toolchain.

We first export the hvcc code to a temporary directory <temp_path>/Source and then copy OwlProgram next to it in <temp_path>/OwlProgram.

Then from the OwlProgram directory we execute the following:
<path_to_our_toolchain/bin/>make -j4 TOOLROOT=<path_to_our_toolchain/bin/> BUILD=../ PATCHNAME=<patch_name> PATCHCLASS=HeavyPatch PATCHFILE=HeavyOWL_owl.hpp load

#13

Hey @dreamer, good to hear that there’s some progress here! I guess you can take a look at what’s in readelf -a patch.elf output and compare it to what you get from OWL’s HVCC version.

I hope it’s not doing something funny like JSON parsing at runtime :wink:

#14

There is no patch.elf created by the linker as it fails exactly at this stage.

Versions shouldn’t really differ that much. And as I said the HeavyTest.pd builds just fine.

#15

Btw if I manually build/run with OwlProgram (same version, namely my fork: https://github.com/Wasted-Audio/OwlProgram ) it does work.

Could it have something to do with gcc versions? That’s the only thing I can definitely point to that is different between my local build and the one in the plugdata toolchain.

In the toolchain we are using arm-gcc v10.2 (this is pinned because of libDaisy) and locally I have v12.2

#16

hmm, not sure what I did now, but my board just gives the infamous E. is unresponsive and gets super hot -_-

#17

Ah right, sorry about that pointless suggestion :wink: If you just want to link a larger patch executable without being able to run it on hardware, you can edit Source/flash.ld and change the line PATCHRAM (rwx) : ORIGIN = 0x2000c000, LENGTH = 144K /* total RAM is 192kb */ to specify LENGTH = 1M or something like that.

Also, we have an option to build patches up to 512kb in size, but they would run only on OWL3 (or Xibeca) boards. This is done by passing PLATFORM=OWL3 when running make commands. It’s not enough to solve this issue (and compiler output is obviously not correct here either way).

Regressions due to toolchain change are certainly possible, I guess you could test the patch generated by OWL’s HVCC with 10.2 GCC to see if it’s also has this issue. Don’t forget to run make realclean; make libs when changing toolchain.

#18

There is really almost no difference in HVCC or Heavylib of the OWL branch or the Wasted Audio branch.

And as said: when I manually build+load it does work (although right now my Lich seems to be completely dead somehow …)

#19

Here’s the readelf outputs for arm-gcc 10 and 12. First one built for OWL3:

https://mrtoasted.com/~dreamer/owl/elf_gcc10.txt
https://mrtoasted.com/~dreamer/owl/elf_gcc12.txt

And for good measure also a build with the linker script patched for OWL2:
https://mrtoasted.com/~dreamer/owl/elf_gcc10_owl2.txt

And the patch used:
https://mrtoasted.com/~dreamer/owl/WindDrone.pd

#20

Ok, so first of all I’ve mentioned wrong linker script, the default one is called owl2.ld and by setting platform to OWL3 you will use owl3.ld which has more RAM for patch code, specifies a different patch address and enables support for double precision FPU. I think you might have figured that out already, just mentioning it to make sure there’s no confusion.

So you say that both files built with GCC10 here are not working? It looks their size is ok and they should fit even in OWL2 SRAM.

Btw, I’ve tried running ./build.sh here and a few things failed, are there any uncommitted changes to this?

#21

Hmmm, when I build with gcc-10.2 now it doesn’t complain about the size … but before it certainly did! (would give the same message about not fitting in region). Too bad my Lich is now dead, so I can’t test anything any more :frowning:

Which parts are failing on that script? I know it’s not a pretty setup, but it should work.

#22

Lich is undying by definition! See how to force bootloader on Lich

Regarding plugdata build errors, first of all it gets a 404 when downloading http://ftp.de.debian.org/debian/pool/main/a/alsa-lib/libasound2_1.1.3-5_amd64.deb .

~/dev/rebeltech/plugdata-heavy-toolchain
--2023-09-20 00:22:31--  http://ftp.de.debian.org/debian/pool/main/a/alsa-lib/libasound2_1.1.3-5_amd64.deb
Resolving ftp.de.debian.org... 141.76.2.4
Connecting to ftp.de.debian.org|141.76.2.4|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-09-20 00:22:32 ERROR 404: Not Found.

ar: /tmp/tmp.HNKZGxHjJb: file format not recognized
tar: data.tar.xz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
cp: cannot stat './usr/lib/x86_64-linux-gnu/libasound.so.2.0.0': No such file or directory

Btw, I’m running Gentoo on aarch64, so the file would be fairly useless :wink:

After compiling OWL libs it thows the following:

~/dev/rebeltech/plugdata-heavy-toolchain/OwlProgram ~/dev/rebeltech/plugdata-heavy-toolchain
In file included from LibSource/ColourScreenPatch.cpp:6:
./Source/PatchProcessor.h:5:10: fatal error: Patch.h: No such file or directory
    5 | #include "Patch.h"
      |          ^~~~~~~~~
compilation terminated.
make: *** [compile.mk:202: Build/ColourScreenPatch.o] Error 1

I guess it doesn’t see it in OwlProgram/LibSource/ for whatever reason.

There was another error, but turns out it was caused by not running it in virtualenv.

#23

No it’s totally fried. When I plug it in it just completely burns up and is totally unresponsive.

#24

Ah, yes the toolchain is not at all tested on arm64 Linux, this will certainly require additional work :slight_smile:

#25

You might test if your OWL digital board is functional by powering it from USB. If not, then you might have to replace it. Otherwise something related to power supply is not working, likely on Lich itself. Might be a dead power regulator, might be something else.

#26

Yeah it does show up, but gets very hot and then after a while disconnects:

[1230610.154759] usb 5-3.4.1: new full-speed USB device number 75 using xhci_hcd
[1230610.256197] usb 5-3.4.1: New USB device found, idVendor=1209, idProduct=dada, bcdDevice= 2.00
[1230610.256202] usb 5-3.4.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[1230610.256204] usb 5-3.4.1: Product: OWL-LICH
[1230610.256206] usb 5-3.4.1: Manufacturer: Rebel Technology
[1230610.256208] usb 5-3.4.1: SerialNumber: 2059388F434E
[1230637.587556] usb 5-3.4.1: urb status -32
[1230637.613827] usb 5-3.4.1: urb status -32
[1230637.635344] usb 5-3.4.1: urb status -32
[1230637.720177] usb 5-3.4.1: urb status -32
[1230637.735507] usb 5-3.4.1: urb status -32
[1230637.755811] usb 5-3.4.1: urb status -32
[1230637.757598] usb 5-3.4.1: urb status -32
[1230637.771745] usb 5-3.4.1: urb status -32
etc ..

Will contact Befaco about replacing or upgrading the board.

#27

Finally got a replacement/upgrade OWL board (v3).
So I’ll circle back on this effort soon-ish.

@antisvin btw before we could support aarch64 Linux we need to find an alternative for the build-anywhere tools that we currently use. This project is for x86 only and is anyway quite outdated. Hasn’t seen activity in years and has quite some old tools.

For aarch64 we indeed would need to make some platform-specific changes as well.

1 Like
#28

I’ve finally had some time to work on this. The main breaking change I had to do to OwlProgram was to disable building the web libraries, but other than that we now have a working prototype for building and flashing OWL2 programs:

2024-12-29_13-51
2024-12-29_13-511053×523 20.6 KB

I have tested it on Linux and macOS x86_64 and mainly Windows needs to be confirmed.

We can export the source (including our included version of OwlProgram), a binary, load the patch into memory or store it into one of 8 slots (not sure how many is feasible, I read somewhere up to 39 slots?)

Eventually we should also add board selection for OWL3, although I’m not sure how best to add the precompiled library. Maybe we need separate copies of OwlProgram for either OWL2 or OWL3?

There is also still the limitation of our Linux toolchain not being available for ARM as we have not found time or resources to replace the build-anywhere tools.

[edit: can’t post more than 3 times consecutively, so I will edit this post instead]

Hmmm, so I was under the impression that if the libraries were built with make libs PLATFORM=OWL2 that they would only work for modules built with PLATFORM=OWL2 as well. However I just built a module with PLATFORM=OWL3 using these libraries and didn’t get any errors and it successfully flashed etc. :thinking:

#29

Glad to hear that something OWL-related still moves forward :wink:

store it into one of 8 slots (not sure how many is feasible, I read somewhere up to 39 slots?)

That’s a compile time option (because some we need to statically allocate metadata in firmware). The default value is 40. It could be that just 39 are usable if we need one entry for storing the patch loaded to memory.

I was under the impression that if the libraries were built with make libs PLATFORM=OWL2 that they would only work for modules built with PLATFORM=OWL2

It’s likely not a problem, because PLATFORM=OWL3 does the following:

  1. allocates patch to a different memory address which allows using patches up to 512kb in size which is not present on STM32F4 MCUs
  2. enables newer version of FPU (sets -mfpu=fpv5-d16 instead of -mfpu=fpv4-sp-d16

The former is irrelevant for building libraries as it takes effect only when linking final patch binary.

Newer FPU supports native 64 bit float maths. It also has a few new instructions, one thing that is definitely relevant is hardware instruction for getting min/max of 2 floats in 32 bit. I think what you describe (build libs for OWL2, patch for OWL2 or OWL3) should be safe to use, because newer FPU mode is a superset of previous.

Maybe you should allow choosing OWL1 platform too, its only effect compared to OWL2 is that it limits patch size (again, happens when linking it). This would prevent users from building a patch that fails on OWL1 hardware.

1 Like
#30

Ah indeed, diff between OWL1 and OWL2 is only the linker script: https://github.com/RebelTechnology/OwlProgram/blob/develop/compile.mk#L61-L75

Thing is I thought that between OWL2 and OWL3 the architecture being different would also mean the libraries wouldn’t be compatible.

I’ll add OWL1 as an option if that means that all the OG users can also make use of this.

So now the main thing will really be testing on Windows … so if anybody is willing to throw themselves at that let me know
:smiley:

#31

Plugdata test builds are here: also add OWL1; build for ALL the OWLs! · Wasted-Audio/plugdata@78afd04 · GitHub
Toolchains are here: run arm-gcc cleanup · Wasted-Audio/plugdata-heavy-toolchain@14eb648 · GitHub

The toolchain needs to be extracted to Documents/plugdata/Toolchain/ or equivalent for the chosen OS.

When manually installing the toolchain you will also have to manually set execution bits for all relevant binaries.
For macOS I also had to strip the quarantine bit on these.