Getting python interface to point to correct library paths

Hi :wave:
I’ve been trying to install acados to work within an ament_cmake environment, setting the ACADOS_INSTALL_DIR to the ament install as given by ${CMAKE_INSTALL_PREFIX} and all includes and libraries get installed in the correct install/include and install/lib folders.

I separately through a cmake add_custom_target() install the python interface, which gets generated in the source directory. But when I go to generate the code, the python library hasn’t updated its paths to the installed directories. How can I ensure that the python interface is installed correctly and works with the installed library paths?

The errors look like this:

acados_solver_angle_control.c:37:10: fatal error: acados_c/ocp_nlp_interface.h: No such file or directory
   37 | #include "acados_c/ocp_nlp_interface.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[4]: *** [<builtin>: acados_solver_angle_control.o] Error 1
Traceback (most recent call last):
  File "/home/peter/ws/build/mpc/solver/angle_control/angle_control/ocp.py", line 158, in <module>
    main()
  File "/home/peter/ws/build/mpc/solver/angle_control/angle_control/ocp.py", line 135, in main
    ocp_solver = AcadosOcpSolver(ocp, json_file='acados_ocp.json')
  File "/home/peter/ws/src/acados/interfaces/acados_template/acados_template/acados_ocp_solver.py", line 975, in __init__
    self.__acados_lib = CDLL(libacados_filepath)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/peter/ws/src/acados/lib/libacados.so: cannot open shared object file: No such file or directory
make[3]: *** [solver/angle_control/CMakeFiles/acados.dir/build.make:61: solver/angle_control/CMakeFiles/acados] Error 1
make[2]: *** [CMakeFiles/Makefile2:300: solver/angle_control/CMakeFiles/acados.dir/all] Error 2
make[1]: *** [Makefile:144: all] Error 2

the OSError is because the lib that it is looking for has been moved to the install/ directory.

Even t_renderer needs to be installed in the src directory for the system to find it. (t_renderer should be downloaded at build time. I spent a week debugging why my build was not finishing in a non-interactive environment, and it turns out the prompt for downloading t-renderer was being hidden by cmake, but never failing)

I think it would be better if there was a more general way to install the python interface, like bundling an egg with all required code and installing into a pythonpath dir.

Just to be clear,
It would be good if everything required for rendering can be installed and find themselves instead of having to set special environment variables. eg the desired install layout would be

install/
  bin/
  include/
  lib/ 
   python-3.8/
     site-packages/ -- here can go the python interface including all the things required for rendering.

  share/ -- alternate location for renderer etc requires ACADOS_SOURCE_DIR env to point here.

I added the following to a package dependent on acados to move stuff out of my acados src and into the
install/share space
But now it cannot render the file. All the input files seem to be present.

 raise Exception(f'Rendering of {in_file} failed!\n\nAttempted to execute OS command:\n{os_cmd}\n\n')
Exception: Rendering of main.in.c failed!

Attempted to execute OS command:
/home/peter/ws/install/share/acados/bin/t_renderer '/home/peter/ws/install/share/acados/interfaces/acados_template/acados_template/c_templates_tera/**/*' 'main.in.c' '/home/peter/ws/build/mpc/solver/angle_control/acados_ocp.json' 'main_angle_control.c'


Same error when running the minimal_example from getting started directory. The rendering command fails.

Exception: Rendering of main.in.c failed!

Attempted to execute OS command:
/home/peter/ws/install/share/acados/bin/t_renderer '/home/peter/ws/install/share/acados/interfaces/acados_template/acados_template/c_templates_tera/**/*' 'main.in.c' '/home/peter/ws/src/acados/examples/acados_python/getting_started/acados_ocp.json' 'main_pendulum_ode.c'

if I use subprocess.run instead of os.system in utils.py line 241… I get a file not found error, not sure which file is not found though.

@FreyJo any idea what this error is?

@FreyJo Okay, that error seems to be when the ACADOS_SOURCE_DIR structure is not

ACADOS_SOURCE_DIR
| 
|-----|
|     include
|-----|
|      lib
|-----|
|      interfaces
|------|
       share

now because I wanted to keep my structure in line with the debian package layout, I had put the interfaces in the share/acados/ folder which broke the compilation.

To fix this I created simlinks into the share/acados/ folder for the lib, include directories. and set the ACADOS_SOURCE_DIR to be pointed at share/acados.

Also added some include paths so that hpipm and blasfeo could find their header files (there’s a funny nesting of includes in the installation layout). I was able to get it to compile in both a native and containerised environment.

I would suggest that perhaps it might be a good idea to put, as a matter of course, the interfaces directory into a share directory on installation, and maybe relax or redesign some of the reliance on the ACADOS_SOURCE_DIR variable and some of the relative paths. That way it may be possible to package this project for distribution as a .deb file or via apt or pip.

Hi Peter,

Let me try to briefly answer a few points:

  1. If you move files and directories in acados around as you please and things don’t work anymore, it is your responsibility.
  2. If you have a proposal to relax the “reliance on the ACADOS_SOURCE_DIR variable and some of the relative paths”, which still works after people move parts in acados around, please go ahead and open a pull request.
  3. Personally, I don’t have a huge interest in restructuring acados.
  4. If you want to make a proposal on how acados should be structured for easier integration, you are welcome to phrase it clearly. This is something that should be discussed for longer before actions are taken IMO, since a lot of people projects would be affected. One suggestion to restructure acados has been made here: Feature/colcon compatible cmake by nielsvd · Pull Request #1 · boschresearch/acados · GitHub
    Maybe you can asses the structure proposed there.
  5. There are currently no efforts “to package this project for distribution as a .deb file or via apt or pip.” as far as I know

Best,
Jonathan

Sure, thats fine, I’ll move this to an issue

hopefully one day installing acados as a dependency (at least in debian systems) will be as easy as
sudo apt install acados-dev
and
pip install acados_template