Sometimes I encounter situations where the residuals of the optimizer “oscillate” between 2 to 3 different points, i.e.
[[1.31082437e+00 1.33229320e+00 7.00000000e-01 8.45138279e-12]
[9.67300283e-01 1.30174425e+00 7.00000000e-01 1.36739902e-13]
[1.31082440e+00 1.33229322e+00 7.00000000e-01 8.45003464e-12]
[9.67300271e-01 1.30174425e+00 7.00000000e-01 1.36854049e-13]
[1.31082445e+00 1.33229325e+00 7.00000000e-01 8.44863366e-12]
[9.67300256e-01 1.30174423e+00 7.00000000e-01 1.36967484e-13]
[1.31082449e+00 1.33229327e+00 7.00000000e-01 8.44732942e-12]
[9.67300241e-01 1.30174424e+00 7.00000000e-01 1.37081023e-13]
[1.31082454e+00 1.33229330e+00 7.00000000e-01 8.44597605e-12]
[9.67300228e-01 1.30174422e+00 7.00000000e-01 1.37192370e-13]]
I am using LINEAR_LS for costs, GAUSS_NEWTON hessian approx. Increasing the levenberg_marquardt parameter doesn’t seem to do much to fix this oscillation, even though it does help in making the residuals smaller:
[[5.00000000e+01 6.35214630e-01 7.00000717e+00 1.02119017e-11]
[5.69069633e+01 8.11401291e-01 1.15773318e+00 4.72147462e-11]
[3.71275859e+00 4.29177039e-01 5.18384888e-01 6.39804303e-13]
[1.37336334e+00 2.84478571e-01 4.17909429e-01 1.04207300e-09]
[4.84865893e-01 1.39154983e-01 4.51455620e-01 3.41555735e-09]
[5.36497286e-01 1.01288530e-01 4.43708424e-01 7.74143062e-09]
[6.22142803e-01 1.89431321e-01 4.19511297e-01 1.27958953e-13]
[5.78156500e-01 2.55680754e-01 7.00000000e-01 6.61404993e-09]
[7.37872305e-01 4.03822181e-01 7.00000000e-01 6.46180649e-10]
[5.51357218e-01 6.10297252e-01 7.00000000e-01 5.89699496e-11]
[1.16469886e+00 4.08550729e-01 7.00000000e-01 5.52833607e-09]
[5.59592811e-01 5.93459420e-01 7.00000000e-01 6.06791948e-09]
[9.77608328e-01 3.29333800e-01 7.00000000e-01 3.54866144e-10]
[8.11654871e-01 5.11505112e-01 7.00000000e-01 4.42114184e-13]
[8.35395165e-01 4.54289893e-01 7.00000000e-01 5.00000000e-15]
[1.08223541e+00 5.74148761e-01 7.00000000e-01 2.10370431e-12]
[8.22944872e-01 4.46674891e-01 7.00000000e-01 3.16985490e-13]
[1.10259330e+00 5.77898367e-01 7.00000000e-01 5.00000000e-15]
[7.65339529e-01 4.46674891e-01 7.00000000e-01 3.69939472e-13]
[1.13434436e+00 5.71884262e-01 7.00000000e-01 1.00472068e-12]
[9.05823331e-01 4.70846383e-01 7.00000000e-01 9.66938774e-12]
[1.10854540e+00 5.99888384e-01 7.00000000e-01 2.72384094e-09]
[9.19316160e-01 5.15197544e-01 7.00000000e-01 6.39233848e-12]
[1.13629183e+00 5.70681213e-01 7.00000000e-01 4.70954480e-13]
[8.84518186e-01 5.15197545e-01 7.00000000e-01 8.21905212e-12]
[1.13965222e+00 5.34252408e-01 7.00000000e-01 4.10702731e-11]
[8.28330034e-01 5.15197545e-01 7.00000000e-01 4.30213478e-09]
[1.14881733e+00 6.69791640e-01 7.00000000e-01 1.48601229e-14]
[7.95752923e-01 5.15197545e-01 7.00000000e-01 1.06550664e-13]
[1.10195697e+00 6.24848384e-01 7.00000000e-01 4.19506642e-10]
[7.62230579e-01 5.15197545e-01 7.00000000e-01 2.12475835e-09]
[1.11799005e+00 5.70983931e-01 7.00000000e-01 1.22162327e-11]
[6.86481491e-01 5.15197545e-01 7.00000000e-01 1.29772540e-10]
[1.17549243e+00 5.28943667e-01 7.00000000e-01 4.60907112e-11]
[6.47010993e-01 5.15197545e-01 7.00000000e-01 2.43542865e-10]
[1.21581259e+00 4.98344095e-01 7.00000000e-01 1.09593117e-14]
[6.68058300e-01 6.05541528e-01 7.00000000e-01 8.00297527e-10]]
Any ideas for why this happens / how to solve this behavior?