Updates to offline compiler

OWL
#1

We’re making some changes to how the offline compilation works, ie compiling with OwlProgram.

Compilation is now split into two parts: libs and patch. Patch is where your code goes. Libs is everything else.

All you have to do is run this command once:
make libs

If you don’t do make libs you will see this error:
arm-none-eabi-gcc: error: Libraries/libowlprg.a: No such file or directory

make libs compiles all the code that is not patch specific, and places the results in two archive files with .a suffix in Libraries. Run this again if you update the project source code, e.g. with a git pull.

To recompile the libs from scratch you can:
make realclean libs

If you don’t have emscripten installed (for building Javascript patches) then make libs will fail with:
/bin/sh: 1: emcc: not found

If you are not using the web target (i.e. not producing Javascript output) then this can be safely ignored, as the main archive will have been produced (in Libraries/libowlprg.a).

Patch compilation works just as before, only it doesn’t build the libs, and make clean only removes the patch specific code. And it’s faster, especially if you use emscripten.

These changes now speed up online compilation by about 10 seconds.

2 Likes
#2

Hello Mars,

I just got my first OWL device a Witch and am happily diving into it. It’s all very straight forward and well engineered I am ver happy with it. I am using Max/gen~ to port some work to it. Web-compiling and uploading works fine.
The only thing I could not find yet is a documentation of how I could compile and upload patches offline. In my studio I don’t have a well working internet connection, so being able to work offline would be a great help.
Could you point me to some resources (i am on MacOs, but if required i could mount a Linux machine as well).

Ideally there would be a step by step guide for dummies :slight_smile: I am not alien to programming, but this is a new deployment chain for me.
Cheers,

Jan

#3

Hi Jan,

You should start here: https://github.com/pingdynasty/OwlProgram/

Using develop branch is recommended. While OWL itself is mostly developed on Linux, there are a few OSX users that build patches offline too. So it should work for you once you setup environment.

One more thing to keep in mind is that Gen~ patches require metadata file to be generated for correct output of CV, I think you would have to write them yourself. @mars probably can share an example of this. When building patches online this gets generated automatically based on parameter widgets configuration.

Stas

#4

Thanks a lot Stas,
I will start to check it out these days.

Cheers Jan

1 Like
#5

Hmm, I’m trying to build a heavy patch (using my hvcc fork) using develop branch and run in to:

/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/lib/thumb/v7e-m+fp/hard/libc.a(lib_a-sbrkr.o): in function `_sbrk_r':
/build/newlib-kGgvwE/newlib-3.3.0/build/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/sbrkr.c:51: undefined reference to `_sbrk'
collect2: error: ld returned 1 exit status
make[1]: *** [compile.mk:159: Build/patch.elf] Error 1
make: *** [Makefile:110: patch] Error 2

Anything I’m missing here? is there something wrong with my gcc?

Something with missing linker flags as mentioned here? → c - Undefined reference to _sbrk - Stack Overflow

#6

Hmm using newer Gcc toolchain is a good idea, but it probably won’t help here. Does your fork use printf function by chance? I think it was leading to some similar error. Otherwise, @mars might want to have a look at generated patch source

#7

Not really planning to move away from debian stable at the moment :wink:

No printf in the OWL generator no. Just the same snprintf() that is also in your HvUtils.h
I don’t use these files that are included in OwlProgram: https://github.com/pingdynasty/OwlProgram/tree/develop/HeavySource

#8

snprintf, like printf, uses heap allocations, that’s probably why you get the sbrk dependency.
Heap allocations in the audio process is a bad idea. Why do you need it, is it just for debug output?

#9

The question is: why do you use it? → https://github.com/pingdynasty/OwlProgram/blob/develop/HeavySource/HvUtils.h#L137

I don’t think this is actually called anywhere in my build, though.

[edit: ok I realize it is called in the OG HvMessage.c → https://github.com/Wasted-Audio/hvcc/blob/feature/owl/hvcc/generators/ir2c/static/HvMessage.c so this could be the cause of this?]

#10

We don’t use hv_snprintf, and it is true that we could hide the macro definition in a header guard. But it would make no difference.

Well yes, that’s why we have our own version of that file.

#11

Check, I understand (kind of). I’ll look at including the overloaded file in my integration.

Also still struggling a bit to understand why/how @owl_param is used in this way, but I may start a different topic about this :slight_smile:

#12

I’ve discussed not using @owl_param with Martin before and suggested to replace it with Heavy’s native primitive. I recall that we’ve eventually figured out that we need owl_param to be able to specify both parameter ID and name (as a string) since that’s what OWL firmware needs.

1 Like
#13

I wonder if there is a better way to retrieve this then to “fork” the workflow like this, though.

In the end my preference would be that patches could run on various targets without too many changes.
But lets discuss that over on my repo perhaps :slight_smile:

#14

If you have any idea how this can be achieved, it would remove a major obstacle from switching to upstream hvcc! But this might require extending hv_param to allow such optional functionality.

#15

There are some other ideas like a @hv_enum we want to implement, so maybe a more general approach could still be useful also for other integrations.

Once we get the tests working again I hope that my branch will be the new “upstream” :wink:

1 Like
#16

Regarding the @owl attribute: specifically what we need is the ability to assign a [send] or [receive] to a specific hardware control, such as Parameter A, or Button 5. And without resorting to ‘magic’ names, like we have to do in gen~ (because gen does not support custom attributes).

So given something like [r Gain @owl A] we know it is a parameter called Gain, assigned to A. The optional arguments for min, max and default values are nice to have but not strictly necessary.

1 Like
#17

So what I have implemented in my fork is an optional “type” to each @hv_param. This can be fully customized in the generator how it’s employed. So far I’ve only used it for [receive] in DPF, like so: https://github.com/Wasted-Audio/hvcc/blob/develop/hvcc/generators/c2dpf/templates/HeavyDPF.cpp#L108
We need to check if this can also be used for [send] (where I think is the main issue for this).

Then maybe we could simply decide to use a type owl (or is it A B etc?) and then treat these parameters in the same way through the jdata dict?

The Daisy integration is very different, but they try to support many different i/o configurations. Your platform has a well defined i/o interface in that sense, right?

#18

I would say that OWL’s IO params are also loosely defined, perhaps even more than what you can expect from Daisy. I.e. some devices could use a param ID as input, while others might end up with the same value as outputs. This is especially important on Magus and some upcoming hardware that is based on MAX11300 chip that can reconfigured on the fly to be input or output. This is toggled by trailing > character in parameter name and that’s also why being able to assign custom string literals is crucial on OWL.

Second difference is that params are also controllable by MIDI, so it’s not uncommon to have more params that are physically present and control them with a controller. Daisy doesn’t really have an equavalent to such generic parameters, it just lets you read knob values by ID and this means that your patch might be unusable on another device.

1 Like
#19

@mars could you remind me how to write the metadata required to assign gen outputs? Like the output for the gate and cv outs for a gen patch to be offline compiled? Also how come this example OWL Patch Library – Rebel Technology has out 6 - what is that on the lich? I think there are only 5 outs no? or is that just the light

#20

There’s some info about metdata in OWL docs and some butt-ugly PHP script that is used on web patcher.

1 Like