Musar zu Eduyot 5:8

And even if one has already entered the machloketh and it is difficult for him to separate from his other friends, still, he should attempt with all of his strength to rescue himself from them. As was done by On ben Peleth who, in the beginning, was of one mind with the men of the machloketh, as written in the Torah, and who, in the end, freed himself from them, as stated in Perek Chelek, Sanhedrin. And, therefore, he was not punished along with them. And he would not pay heed to the words of his yetzer and its claims that it would not befit him to stop in the middle and not consummate the triumph. For it is better that one be shamed in this world before a few individuals and not be shamed in the world to come before all of the heavenly assembly. And Chazal said (Eduyoth 5a): "Better that one be called a fool all of his days than be wicked for one moment before the L-rd."