Vxworks 32bit system, PPC, Diab Compiler, Embedded application problems

Hi there,

I have successfully cross-compiled ACADOS for VxWorks 6.9, a 32-bit system running on PPC CPU. Here are the settings

jietu1
jietu3
jietu2

So

  • The target is set to GENERIC

  • EXT_DEP is turned off

  • LA uses REFERENCE

  • MF uses COLMAJ

The compilation is successful, despite a few warnings. Then I successfully compiled locally the c code of the bicycle model generated from the Python interface. For simplicity, I tested the integrator first.

When I try to run the executable, I got the following output
jietu4
So basically the code is not working. Ok, I start debug, and found that the error came from the struct mem in the integrator.
jietu5
When I try to give values to the mem->time_ad, mem->time_la and mem->time_sim, the error shows. Then I found that these three double variables are not created, or their memory is not assigned properly.
jietu6
Finally, I simply comment these three lines, and the integrator works!

My question is, why is the case? There seems memory alignment or assignment issues, but weirdly everything works fine on Ubuntu 64bit system, and on Speedgoat.

To further locate the problem, I test the OCP example as well, and the code stuck at model->scaling=1.0
The similarity of these two tests is that: both sim_irk_memory and model struct contains a few double pointers and a few double variables. The pointers work fine, but the double variables are crashed. I guess, this is the problem of align_char_to(8, &c_ptr), or any thing related to this?

Thank you for your advice.

Yutao

A quick update:

By adding memory alignment before the introduction of the mem, the error disappears


Hence, it seems that an initial memory alignment is necessary in my case.

Although this solves my issue for the integrator test, further tests on OCP example show that I need to modify dozens of functions, which makes this workaround tedious. Is there better way to to it?

Yutao

Hi Yutao,

I tested for 32 bit compatibility over a year ago on a 32 bit Raspberry Pi, and wrote down this memory management guide:
https://docs.acados.org/developer_guide/index.html#memory-management-in-acados

It seems that on your architecture you get “less aligned” memory from the system and applying the guide more rigorously is required.

I hope you can fix some of the issues and make a PR.

For reference:

Best,
Jonathan

Hi Jonathan,

Thank you for the reply. The guide you gave is very helpful.

However, after adapting the memory alignment guide to our architecture, we still got some errors. The errors indicate that HPIPM structs have memory problems, and this kind of struct is not explained in the guide you provided.

Specifically, we found that the first error lied in the function

ocp_qp_hpipm_memory_assign

and in which the sub-struct

d_ocp_qp_ipm_ws

is created by calling

d_ocp_qp_ipm_ws_create

We tried to align before/after this call, but still got errors. This means that

mem->hpipm_workspace

is not created succussfully.

However, this function is very complicate and given in external/hpipm, hence we don’t think it’s a good idea to change it.

What should/can I do? Any suggestions?

Yutao

Hi Yutao,

Can you try to add an align here?
i.e. before d_ocp_qp_ipm_ws, which has some doubles at the beginning.

Also, HPIPM has been successfully tested on a 32bit ARM, but it might be that there are issues left for PPC. It might be also related to the fact that PPC is big endian is big endian by default.

I hope this helps you to fix it and that you contribute the fix!

Best,
Jonathan

Hi Jonathan,

We tried your suggestion but that does not work.

However, we went through the code deeper and find out mem->hpipm_workspace->stat address can be printed out, and mem->hpipm_workspace->stat_m has a value of 17.

When we tried to print mem->hpipm_workspace->stat[0], the program stoped. Hence, it is obvious that the memory of this vector is not assigned properly.

We then checked this vector in x_ocp_qp_ipm.c (in function void OCP_QP_IPM_WS_CREATE), and found the codes below

// double/float stuff
	REAL *d_ptr = (REAL *) sv_ptr;
	workspace->stat = d_ptr;
	int stat_m = 17;
	d_ptr += stat_m*(1+arg->stat_max);

So, stat_m=17 is confirmed, and clearly the vector “stat” has been assigned memory here, but somehow not successfully when returned to function void *ocp_qp_hpipm_memory_assign

Finally, we tried a very direction solution, i.e. adding manually the memory for this vector in the function void *ocp_qp_hpipm_memory_assign as

assign_and_advance_double(17*(1+opts->stat_max), &ipm_workspace->stat, &c_ptr);

So the final function of void *ocp_qp_hpipm_memory_assign becomes

Magically, the program worked

Now I’m not sure if this can be a PR (does it work for other architectures?), and why the memory of a single vector is not assigned inside HPIPM create function correctly so that we have to do it manually in ACADOS create function.

Cheers,

Yutao

I am also struggling with memory erros on a QNX with PPC machine (dSPACE) and tried this fix. However I get the following error, when applying your fix. Do we have different code versions?

‘ocp_qp_hpipm_opts {aka struct ocp_qp_hpipm_opts_}’ has no member named ‘stat_max’
assign_and_advance_double(17*(1+opts->stat_max), &ipm_workspace->stat, &c_ptr);

Maybe you have an idea?

Thanks,
Albert

Hi albert,

I made a mistake. The correct fix should be

assign_and_advance_double(mem->hpipm_workspace->stat_m*(1+opts->hpipm_opts->stat_max), &mem->hpipm_workspace->stat, &c_ptr);

Yours,

Yutao

Hi Yutao,

thanks for the quick answer!

I made the change and it compiles, at runtime the assertion in the following line does throw an error now.

In function ocp_qp_hpipm_memory_assign – C:/Users/praktika/acados/acados/ocp_qp/ocp_qp_hpipm.c:218 (char *) raw_memory + ocp_qp_hpipm_memory_calculate_size(config_, dims, opts_) >= c_ptr – assertion failed

Was this the case for you as well? It shouldnt be like this, right?

Albert

Hi Albert,

You also need to fix accordingly in the function

ocp_qp_hpipm_memory_calculate_size

So, basically, you need to add this additional memory when calculating the size of the memory. Check the link Jonathan share above.

Best,

Yutao

In case of 32-bit systems and 64-bit FP, it may be that in HPIPM the memory before assigning stat is not aligned to 8-bytes boundaries.
In this commit

I add an explicit alignment before it, can you check if this makes the job for you?

1 Like

Hi Gianluca,

I have tested your commit and it worked. The results are correct.

Thank you very much.

Just be curious, will these additional alignments affect 64-bit systems (Ubuntu 64 on a PC)?

Yutao

This extra alignment has no effect on 64-bit systems, other than using a tiny bit more memory.

I implemented the same fix also for the dense solver and the tree-structure ocp qp solver.

Thanks again for spotting and the precise hints to locate the issue.

1 Like