MHE solver always returns sqp_iter=0

Hi guys,

I am using acados with Python interface to solve a MHE simple test program using my own model instead of export_pendulum_ode_model().
I tried to replicate as much as possible the structure of minimal_example_mhe.py.
The first section of my program, that implements an OCP to generate data to be fed to the MHE, works without problems. The following MHE section instead produces totally empty arrays after executing
status = acados_solver_mhe.solve().
In addition, with acados_solver_mhe.get_stats('sqp_iter'), I checked that the MHE solver returns after just 0 SQP iteration. No error messages are produced.

The only difference compared to the original minimal_example_mhe.py is that I have an output y=h(x,u) that is a very complex function of the status x and so I was forced to generate the nonlinear cost_y_expr directly in the export_mhe_ode_model and add “model.cost_y_expr = cost_y_expr” in it at the end, instead of adding “model.cost_y_expr = cost_y_expr” inside export_mhe_solver.

I checked everything to try to solve this but I didn’t find any other difference with the original example.
I thank you in advance for your support.

Cheers,
Walter

Hi Walter,

it is hard to tell from this what is going on.
One thing that could lead to 0 SQP iterations, is that the initialization is already the solution.
I would try something like

acados_solver_mhe.print_statistics()

to see if the residuals are evaluated initially, and if they are small.

However, this does not explain why you get empty arrays from the MHE solver.
Maybe check in the MHE .json file if the dimensions are correct.

Cheers,
Jonathan

Hi Jonathan,

thank you for your prompt reply.
As I am using an ACADOS version of March 2020, I don’t have the function print_statistics in AcadosOcpSolver. I am quite afraid to upgrade it to the last version because when I installed it the first time it took me quite a lot to have it up and running in PyCharm. Is there an easy way to just modify the AcadosOcpSolver file I already have to insert the print_statistics function?
Thanks

Cheers,
Walter

Hi Jonathan,

I reinstalled Acados to the latest release.
It seems that it still works fine in Pycharm.
minimal_example_closed_loop.py runs to completion without errors.
Now I can’t run successfully anymore minimal_example_mhe.py.
This is the screenshot I got

Is there something missing?
Many thanks

Cheers,
Walter

Hi Jonathan,

problem now solved.
I recompiled with the following instructions:

and now minimal_example_mhe.py runs successfully.
I will try now to run my mhe program with acados_solver_mhe.print_statistics()

Cheers,
Walter

Hi Jonathan,

hereafter the screenshot I got with acados_solver_mhe.print_statistics()

Cheers,
Walter

So you were missing -DACADOS_WITH_QPOASES?
I can try to have another look at the actual issue of this topic next week.
If you can share the source code with me, that might be the easiest.

Thanks a lot Jonathan.
I will try for next week to share some code snippets with you.

Have a good weekend
Walter

Hy Jonathan,

I copied in pastebin three parts of my MHE program:

  1. MAIN.MHE https://pastebin.com/Ukjdy5HZ
  2. export_mhe_model https://pastebin.com/UzLFLzLp
  3. export_mhe_solver https://pastebin.com/arfgPEgk

I can run the program in debugging mode and paste the contents of the variables in critical breakpoints if you need.

Thanks a lot for your support

Cheers,
Walter

Hi Jonathan,

I copied in pastebin the acados_ocp.json file as well.

Hi Walter,

I just had a brief look at the json file.

The following line seems not valid:
"Vu""Vx""Vx_e": [],
Is this actually the generated json file?

Some more questions to trouble shoot this issue…

Do the libraries for the solver compile correctly?
I.e. when you do:

acados_solver_mhe = export_mhe_solver_Li_SPM_1(model_mhe, N, h, Q_mhe, Q0_mhe, R_mhe)

Moreover, are the model.name different for the MHE and MPC solver?

Maybe try again after removing the whole c_generated_code folder.

Did you try to get the values from the MHE solver?
Do they have the correct dimensions?
Are they zero or matching your initialization or have they been changed by the solver?

Best,
Jonathan

Hi Jonathan,

The generated json file is the one at the bottom of the pastebin page “RAW paste data”.
The libraries for the solver compile correctly.

/home/walter/anaconda3/envs/NMPC_acados/bin/python /snap/pycharm-community/current/plugins/python-ce/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 36335 --file /home/walter/acados/examples/acados_python/getting_started/HYPEMLIFE_NMPC_test/minimal_Li_SPM_mhe_1.py
pydev debugger: process 17228 is connecting

Connected to pydev debugger (build 202.7660.27)
rm -f libacados_ocp_solver_Lithium_SPM.so
rm -f acados_solver_Lithium_SPM.o
( cd Lithium_SPM_model; gcc -fPIC -std=c99  -c   Lithium_SPM_impl_dae_fun.c Lithium_SPM_impl_dae_fun_jac_x_xdot_z.c Lithium_SPM_impl_dae_jac_x_xdot_u_z.c)
( cd Lithium_SPM_constraints; gcc -fPIC -std=c99  -c   Lithium_SPM_constr_h_fun_jac_uxt_zt.c Lithium_SPM_constr_h_fun.c)
gcc -fPIC -std=c99  -c acados_solver_Lithium_SPM.c -I /home/walter/acados/include/blasfeo/include/ -I /home/walter/acados/include/hpipm/include/ \
-I /home/walter/acados/include -I /home/walter/acados/include/acados/ \

gcc -fPIC -std=c99  -shared -o libacados_ocp_solver_Lithium_SPM.so  Lithium_SPM_constraints/Lithium_SPM_constr_h_fun_jac_uxt_zt.o Lithium_SPM_constraints/Lithium_SPM_constr_h_fun.o acados_solver_Lithium_SPM.o  Lithium_SPM_model/Lithium_SPM_impl_dae_fun.o Lithium_SPM_model/Lithium_SPM_impl_dae_fun_jac_x_xdot_z.o Lithium_SPM_model/Lithium_SPM_impl_dae_jac_x_xdot_u_z.o \
-L -l \
-L /home/walter/acados/lib -lacados -lhpipm -lblasfeo \
-lm \

rm -f libacados_ocp_solver_mhe_Lithium_SPM.so
rm -f acados_solver_mhe_Lithium_SPM.o
( cd mhe_Lithium_SPM_model; gcc -fPIC -std=c99  -c   mhe_Lithium_SPM_impl_dae_fun.c mhe_Lithium_SPM_impl_dae_fun_jac_x_xdot_z.c mhe_Lithium_SPM_impl_dae_jac_x_xdot_u_z.c)
( cd mhe_Lithium_SPM_cost; gcc -fPIC -std=c99  -c   mhe_Lithium_SPM_cost_y_fun.c mhe_Lithium_SPM_cost_y_fun_jac_ut_xt.c mhe_Lithium_SPM_cost_y_hess.c)
gcc -fPIC -std=c99  -c acados_solver_mhe_Lithium_SPM.c -I /home/walter/acados/include/blasfeo/include/ -I /home/walter/acados/include/hpipm/include/ \
-I /home/walter/acados/include -I /home/walter/acados/include/acados/ \

gcc -fPIC -std=c99  -shared -o libacados_ocp_solver_mhe_Lithium_SPM.so  mhe_Lithium_SPM_cost/mhe_Lithium_SPM_cost_y_fun.c mhe_Lithium_SPM_cost/mhe_Lithium_SPM_cost_y_fun_jac_ut_xt.c mhe_Lithium_SPM_cost/mhe_Lithium_SPM_cost_y_hess.c acados_solver_mhe_Lithium_SPM.o  mhe_Lithium_SPM_model/mhe_Lithium_SPM_impl_dae_fun.o mhe_Lithium_SPM_model/mhe_Lithium_SPM_impl_dae_fun_jac_x_xdot_z.o mhe_Lithium_SPM_model/mhe_Lithium_SPM_impl_dae_jac_x_xdot_u_z.o \
-L -l \
-L /home/walter/acados/lib -lacados -lhpipm -lblasfeo \
-lm \


iter	res_stat	res_eq		res_ineq	res_comp	qp_stat	qp_iter
0	0.000000e+00	0.000000e+00	0.000000e+00	0.000000e+00	0	0


None
difference |x0_est - x0_bar| 298.15111766350964
difference |x_est - x_true| 1001.3855074354593

Process finished with exit code 0

The model.name for MPC solver is export_Li_SPM_opc_model_4() and for the MHE solver is export_Li_SPM_mhe_model_4().

Did you try to get the values from the MHE solver?
Do you mean simXest, simWest? They are all zeros.

Thank you
Cheers,
Walter

Hi Jonathan,

is there a way to see what happens when acados_solver_mhe.solve() is executed?
Why the residuals of the first iteration are all zero?

BR,
Walter

Hi Walter,

from the compilation output, everything seems fine.

Setting print_level and getting values from the solver and the print_statistics are usually enough for users.
However, at this point, it seems like one has to dig deeper.

Where does the solver terminate?
Why does it before updating the variables, which should at least be done once?

For OCP solvers, I would first compile and run the associated generated main C file.
And run it with a debugger.
However, I am not sure if this main file needs to be adapted for the MHE case.

Alternatively, one can add prints in the SQP source code to see what is going on…

Best,
Jonathan

Hi Jonatahan,
thanks for your suggestion.
Before start digging in acados C files (this would be quite a daunting task to me!) I want to try one thing.
Could it be that the solver terminates at iter=0 because, during the minimization, the state function evolution f(x,u,w) becomes infinite? Differently from the OPC solver, in the MHE solver I didn’t put bounds on x variables and I know that my model equations can become infinite if x goes outside a specified range.
I’ll let you know the results of this test.

BR,
Walter

Hi Jonathan,

with bounds on x and x0 now the MHE solver terminate successfully.

/home/walter/anaconda3/envs/NMPC_acados/bin/python /home/walter/acados/examples/acados_python/getting_started/HYPEMLIFE_NMPC_test/minimal_Li_SPM_mhe_1.py
rm -f libacados_ocp_solver_Lithium_SPM.so
rm -f acados_solver_Lithium_SPM.o
( cd Lithium_SPM_model; gcc -fPIC -std=c99 -c Lithium_SPM_impl_dae_fun.c Lithium_SPM_impl_dae_fun_jac_x_xdot_z.c Lithium_SPM_impl_dae_jac_x_xdot_u_z.c)
( cd Lithium_SPM_constraints; gcc -fPIC -std=c99 -c Lithium_SPM_constr_h_fun_jac_uxt_zt.c Lithium_SPM_constr_h_fun.c)
gcc -fPIC -std=c99 -c acados_solver_Lithium_SPM.c -I /home/walter/acados/include/blasfeo/include/ -I /home/walter/acados/include/hpipm/include/
-I /home/walter/acados/include -I /home/walter/acados/include/acados/ \

gcc -fPIC -std=c99 -shared -o libacados_ocp_solver_Lithium_SPM.so Lithium_SPM_constraints/Lithium_SPM_constr_h_fun_jac_uxt_zt.o Lithium_SPM_constraints/Lithium_SPM_constr_h_fun.o acados_solver_Lithium_SPM.o Lithium_SPM_model/Lithium_SPM_impl_dae_fun.o Lithium_SPM_model/Lithium_SPM_impl_dae_fun_jac_x_xdot_z.o Lithium_SPM_model/Lithium_SPM_impl_dae_jac_x_xdot_u_z.o
-L -l
-L /home/walter/acados/lib -lacados -lhpipm -lblasfeo
-lm \

rm -f libacados_ocp_solver_mhe_Lithium_SPM.so
rm -f acados_solver_mhe_Lithium_SPM.o
( cd mhe_Lithium_SPM_model; gcc -fPIC -std=c99 -c mhe_Lithium_SPM_impl_dae_fun.c mhe_Lithium_SPM_impl_dae_fun_jac_x_xdot_z.c mhe_Lithium_SPM_impl_dae_jac_x_xdot_u_z.c)
( cd mhe_Lithium_SPM_cost; gcc -fPIC -std=c99 -c mhe_Lithium_SPM_cost_y_fun.c mhe_Lithium_SPM_cost_y_fun_jac_ut_xt.c mhe_Lithium_SPM_cost_y_hess.c)
gcc -fPIC -std=c99 -c acados_solver_mhe_Lithium_SPM.c -I /home/walter/acados/include/blasfeo/include/ -I /home/walter/acados/include/hpipm/include/
-I /home/walter/acados/include -I /home/walter/acados/include/acados/ \

gcc -fPIC -std=c99 -shared -o libacados_ocp_solver_mhe_Lithium_SPM.so mhe_Lithium_SPM_cost/mhe_Lithium_SPM_cost_y_fun.c mhe_Lithium_SPM_cost/mhe_Lithium_SPM_cost_y_fun_jac_ut_xt.c mhe_Lithium_SPM_cost/mhe_Lithium_SPM_cost_y_hess.c acados_solver_mhe_Lithium_SPM.o mhe_Lithium_SPM_model/mhe_Lithium_SPM_impl_dae_fun.o mhe_Lithium_SPM_model/mhe_Lithium_SPM_impl_dae_fun_jac_x_xdot_z.o mhe_Lithium_SPM_model/mhe_Lithium_SPM_impl_dae_jac_x_xdot_u_z.o
-L -l
-L /home/walter/acados/lib -lacados -lhpipm -lblasfeo
-lm \

iter res_stat res_eq res_ineq res_comp qp_stat qp_iter
0 6.000006e+05 1.055455e+00 4.500000e+01 0.000000e+00 0 0
1 2.165226e+01 7.577910e-02 2.842171e-14 5.316761e-13 0 27
2 1.453881e-03 7.021768e-04 2.842171e-14 5.755516e-13 0 16
3 9.922849e-10 3.003044e-08 2.842171e-14 5.149805e-13 0 16
4 2.315870e-10 5.684342e-14 2.309264e-14 5.149873e-13 0 16

None
difference |x0_est - x0_bar| 1.520932051175864e-09
difference |x_est - x_true| 7.073162972453633e-08

Process finished with exit code 0

Thank you again Jonathan

Cheers,
Walter

Hi Walter,

great that you solved it without digging into the C code!
It seems reasonable to have these bounds in your problem.

I still would have guessed, that the infinity values from the linearization go to the QP solver, which then throws an error, instead of returning None status.

Anyway, great it works for you now!

Cheers,
Jonathan