Segmentation Fault importing acados_template from Python 3.5

PRoque · November 12, 2020, 9:58am

Hi everyone,

As suggest, @zanellia , you seem the one to reach.

I am setting up acado to run with python, but I ran into the following problem:
After doing all steps successfully and make/make install to the default directory, and adding it to the LD_LIBRARY_PATH, I run into a Segmentation fault on the minimal working example.

I am using Python 3.5 on Ubuntu 16.04

My acado build directory:
/home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/build

My default installation path:
– Install configuration: “Release”
– Up-to-date: /home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/lib/libacados.so

My LD_LIBRARY_PATH, on .zshrc:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/lib"

Module installation:
➜ acados git:(master) ✗ pip3 install -e interfaces/acados_template
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/interfaces/acados_template
Requirement already satisfied: numpy in /home/roque/.local/lib/python3.5/site-packages (from acados-template==0.1.4.dev80+gdfdc073.d20201112) (1.18.5)
Requirement already satisfied: scipy in /home/roque/.local/lib/python3.5/site-packages (from acados-template==0.1.4.dev80+gdfdc073.d20201112) (1.4.1)
Requirement already satisfied: casadi>=3.5.1 in /home/roque/.local/lib/python3.5/site-packages (from acados-template==0.1.4.dev80+gdfdc073.d20201112) (3.5.5)
Requirement already satisfied: matplotlib in /home/roque/.local/lib/python3.5/site-packages (from acados-template==0.1.4.dev80+gdfdc073.d20201112) (3.0.3)
Requirement already satisfied: future-fstrings in /home/roque/.local/lib/python3.5/site-packages (from acados-template==0.1.4.dev80+gdfdc073.d20201112) (1.2.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/roque/.local/lib/python3.5/site-packages (from matplotlib->acados-template==0.1.4.dev80+gdfdc073.d20201112) (1.1.0)
Requirement already satisfied: python-dateutil>=2.1 in /home/roque/.local/lib/python3.5/site-packages (from matplotlib->acados-template==0.1.4.dev80+gdfdc073.d20201112) (2.8.1)
Requirement already satisfied: cycler>=0.10 in /home/roque/.local/lib/python3.5/site-packages (from matplotlib->acados-template==0.1.4.dev80+gdfdc073.d20201112) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/lib/python3/dist-packages (from matplotlib->acados-template==0.1.4.dev80+gdfdc073.d20201112) (2.0.3)
Requirement already satisfied: tokenize-rt>=3; python_version < “3.6” in /home/roque/.local/lib/python3.5/site-packages (from future-fstrings->acados-template==0.1.4.dev80+gdfdc073.d20201112) (3.2.0)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from kiwisolver>=1.0.1->matplotlib->acados-template==0.1.4.dev80+gdfdc073.d20201112) (20.7.0)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.1->matplotlib->acados-template==0.1.4.dev80+gdfdc073.d20201112) (1.10.0)
Installing collected packages: acados-template
Attempting uninstall: acados-template
Found existing installation: acados-template 0.1.4.dev80+gdfdc073.d20201112
Uninstalling acados-template-0.1.4.dev80+gdfdc073.d20201112:
Successfully uninstalled acados-template-0.1.4.dev80+gdfdc073.d20201112
Running setup.py develop for acados-template
Successfully installed acados-template

What happens when I run the example:
getting_started git:(master) ✗ python3.5 minimal_example_closed_loop.py
[1] 4560 segmentation fault (core dumped) python3.5 minimal_example_closed_loop.py

Is there something really wrong with any of these steps? Or what am I missing?

Cheers,
Pedro

FreyJo · November 12, 2020, 2:24pm

Hey,

Let me ask some questions to get into what is going on:

Personally I use the CMake build system, it is also the one tested continuously. Can you try using it instead of Make?
https://docs.acados.org/installation/#cmake
For Make you might have to select the BLASFEO and HPIPM targets manually. Which ones did you use?
Which version of acados are you using?

Best,
Jonathan

PRoque · November 12, 2020, 3:27pm

Thanks Frey!

So, I used cmake .. to configure the build, and then used make install. Popping to the last question, I just cloned your master repo, so the latest would be my guess.

Thanks a lot!
Pedro,

FreyJo · November 12, 2020, 3:29pm

So, does it work with CMake?

PRoque · November 12, 2020, 4:11pm

No no, it doesn’t, the problem I’m facing is actually using cmake, and then make install. But I don’t have any error or warning during the installation, just that segfault when trying it

PRoque · November 13, 2020, 9:26am

@FreyJo I essentially followed everything in the Python interfaces section:

I got no error whatsoever in any of the commands, so that’s why I found it strange to segfault. Any clue?

Edit: some more dubugging: it seems that python can find AcadosOcp, but when I try to go to definition of AcadosOcpSolver, AcadosSimSolver, it can’t find it. The script exits with segfault here:
from acados_template import AcadosOcp, AcadosOcpSolver, AcadosSimSolver

Edit2: I just did the same procedure in Ubuntu 18.04, and it was successful. So far, I can’t see why it wouldn’t work on Ubuntu 16, since everything compiles correctly. Any guess?

PRoque · November 13, 2020, 2:14pm

Yet more information: upon inspecting with ldd, got this:

➜  lib git:(master) ldd libacados.so 
	linux-vdso.so.1 =>  (0x00007ffe47d9b000)
	libqpOASES_e.so.3.1 => not found
	libhpipm.so => not found
	libblasfeo.so => not found
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff8136be000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff8132f4000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff813c35000)

However, these libraries exist under <acados_root>/lib

Edit: even after fixing it with
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"<acados_root>/lib"
where I replaced the root with the proper folder, it still gives segmentation fault and Tera is never invoked.

So, essentially, is this an issue with Ubuntu 16.04 LTS in general?

FreyJo · November 13, 2020, 4:48pm

Hi @PRoque

The script exits with segfault here:
from acados_template import AcadosOcp, AcadosOcpSolver, AcadosSimSolver

Are you sure about this?
In this case, it is a Python related problem.

even after fixing it with
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:“<acados_root>/lib”
where I replaced the root with the proper folder, it still gives segmentation fault and Tera is never invoked.

If it Segfaults before trying to create the AcadosOcpSolver object, the problem should not be missing libraries.

Can you please try to run an acados C example?
To do so, use CMake with -DACADOS_EXAMPLES=ON and run for example ./build/examples/c/wind_turbine_nmpc_example.
This would rule out if it is a problem in the Python interface or in the C part.

I don’t expect the issue to be related to the ubuntu version.

What would be interesting, if C examples dont work, are the BLASFEO & HPIPM targets, i.e. this part of the CMake output:

-- Build type is Release
-- ACADOS_WITH_OPENMP: OFF
-- Using linear algebra: HIGH_PERFORMANCE
-- Using matrix format: PANELMAJ
-- Using external BLAS: 0
-- Testing target X64_INTEL_HASWELL: assembly compilation [success]
-- Testing target X64_INTEL_HASWELL: assembly run [success]
-- Testing target X64_INTEL_HASWELL: intrinsic compilation [success]
-- Testing target X64_INTEL_HASWELL: intrinsic run [success]
-- Detected target X64_INTEL_HASWELL
-- Using BLASFEO path: 
-- Installation directory: /home/oj/acados
-- Found Eigen3: /usr/include/eigen3 (Required is at least version "2.91.0") 
--  
-- Target: BLASFEO is X64_AUTOMATIC, HPIPM is X64_AUTOMATIC
-- Linear algebra: HIGH_PERFORMANCE

Cheers,
Jonathan

PRoque · November 13, 2020, 5:08pm

Thanks Jonathan! Some progress, the C examples, indeed, do work. So the issue might be with the installation of the template.

My pip3 version is 20.2.4. This is what happens in a Python 3.5 shell:
➜ ~ python3
Python 3.5.2 (default, Oct 7 2020, 17:19:02)
[GCC 5.4.0 20160609] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import acados_template
[1] 16621 segmentation fault (core dumped) python3
➜ ~

So can the problem be python3.5 pip ? Should I try to install Python3.6 and its pip3, and try with it?

Pedro,

PRoque · November 13, 2020, 5:23pm

More debugging: the issue seems to be Python3.5 and its pip - I tried with python3.6 and its pip and the installation went fine - although I still have issues afterwards.

rm -f libacados_ocp_solver_pendulum_ode.so
rm -f acados_solver_pendulum_ode.o
( cd pendulum_ode_model; gcc -fPIC -std=c99  -c   pendulum_ode_expl_ode_fun.c pendulum_ode_expl_vde_forw.c)
gcc -fPIC -std=c99  -c acados_solver_pendulum_ode.c -I /home/roque/.local/lib/include/blasfeo/include/ -I /home/roque/.local/lib/include/hpipm/include/ \
-I /home/roque/.local/lib/include -I /home/roque/.local/lib/include/acados/ \

acados_solver_pendulum_ode.c:38:32: fatal error: acados/utils/print.h: No such file or directory
compilation terminated.
Makefile:87: recipe for target 'ocp_solver' failed
make: *** [ocp_solver] Error 1
Traceback (most recent call last):
  File "minimal_example_closed_loop.py", line 104, in <module>
    acados_ocp_solver = AcadosOcpSolver(ocp, json_file = 'acados_ocp_' + model.name + '.json')
  File "/home/roque/.local/lib/python3.6/site-packages/acados_template/acados_ocp_solver.py", line 693, in __init__
    self.shared_lib = CDLL(self.shared_lib_name)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: c_generated_code/libacados_ocp_solver_pendulum_ode.so: cannot open shared object file: No such file or directory

FreyJo · November 16, 2020, 9:02am

Hi @PRoque,

good to know there is an issue with Python 3.5. I’ll try to look into it.

The issue you are now having with Python 3.6, seems to be a problem with the include path.

acados_solver_pendulum_ode.c:38:32: fatal error: acados/utils/print.h: No such file or directory

If your build folder is still this
/home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/build
the include paths should be (by default)
/home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/include/acados
/home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/include/blasfeo/include
/home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados/include/hpipm/include

Or did you change the build system to put the headers here:
/home/roque/.local/lib/include?

Best,
Jonathan

PRoque · November 16, 2020, 9:44am

Hi @FreyJo,

I didn’t change the include paths, and I found it wierd for to include my .local instead of the build directory too. Something fishy with the CMake config?

I’ll try to add those manually to my C_INCLUDE_PATH and see if I get past the problems.
Update: it fixes includes but then lacados -lhpipm -lblasfeo , which should be discoverable from LD_LIBRARY_PATH…

Pedro,

PRoque · November 16, 2020, 9:56am

More on this: if I set my install dir to:
-DACADOS_INSTALL_DIR=/home/roque/.local

and add the above includes, and add
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/roque/.local

it still tries to include
-L /home/roque/.local/lib/lib -lacados -lhpipm -lblasfeo

which makes no sense - it should upmost go one level down. Should I look into the Cmake? My Cmake version is 3.5.1

FreyJo · November 16, 2020, 10:59am

Hi Pedro,

Do you use some Python virtual environment?
There are known issues with conda, miniconda and Pycharm.
See the section on “virtualenv” here: https://docs.acados.org/interfaces/index.html#python-templates

I think this function is not working properly in this case:

github.com

acados/acados/blob/dfdc0734c8eeaf18705bebe6588d5dc88c1fc4ba/interfaces/acados_template/acados_template/utils.py#L44-L50


def get_acados_path():
    ACADOS_PATH = os.environ.get('ACADOS_SOURCE_DIR')
    if not ACADOS_PATH:
        acados_template_path = os.path.dirname(os.path.abspath(__file__))
        acados_path = os.path.join(acados_template_path, '../../../')
        ACADOS_PATH = os.path.realpath(acados_path)
    return ACADOS_PATH

PRoque · November 16, 2020, 11:42am

Hi @FreyJo,

No, no virtual environments, just plain .zshrc and the only thing I source is two ROS workspaces.

I can try to see if they have anything to do with that, but in Ubuntu 18 I also source those, so they shouldn’t be a problem.

Doing an echo on ACADOS_SOURCE_DIR returns blank too, so perhaps that is a problem?

FreyJo · November 16, 2020, 12:56pm

I would try to do export ACADOS_SOURCE_DIR=<your acados path>
where for you the path should be /home/roque/reswarm_ws/src/reswarm_dmpc/lib/acados.

PRoque · November 16, 2020, 1:52pm

It’s a miracle

Alright, managed to run it with Python 3.5 . To make things simpler, here’s what I did:

Cloned acados to /home/roque/Git/acados
compiled with
cmake -DACADOS_WITH_QPOASES=ON -DACADOS_EXAMPLES=ON ..
installed acados_template as suggested, but without “-e” (although that shouldn’t matter)
Export the lib path
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/roque/Git/acados/lib
Exported the ACADOS_SOURCE_DIR (this one is a must):
export ACADOS_SOURCE_DIR=/home/roque/Git/acados
ran the example
cd /home/roque/Git/acados/examples/acados_python/getting_started
python3 minimal_example_closed_loop.pyp

So, the function you mention above, perhaps instead of trying to fill the ACADOS_PATH with some guess, it should actually tell the user to export it. I’ve added a function to my .zshrc to export both the lib and source dirs now, so whenever I need to run it I just do ldacados , and everything is setup.

In particular, in my .zshrc, I have:
ldacados(){
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/roque/Git/acados/lib
export ACADOS_SOURCE_DIR=/home/roque/Git/acados
}

I’ll try to do some OCPs now with it with CasADi models I have.

FreyJo · November 16, 2020, 2:33pm

Hi Pedro,

I am glad to hear that it works for you now!

Just keep in mind you will have to reinstall when updating acados.

Makes sense, I will at least add a warning there!

Thanks for sharing how you resolved your issue!

Good luck with that!

PRoque · November 17, 2020, 8:12pm

@FreyJo posted a few questions related to this here:

Hopefully I can build a set of examples that then could be added to your docs/database. All input is welcome!

1111 · May 11, 2022, 2:28am

I meet the same problem when installing the python Interfaces. I install acados in ubuntu18.06 with python 3.7. And i try to run an acados C example, it works.
But when i run python3 minimal_example_closed_loop.pyp , it get errors as follows.