So I was in the process of renovating this stunning Remington Portable Model 5 and noticed this metal lever at the back. After being puzzled for a bit, I begun tinkering with it. I traced it on the bottom of the typewriter and found out it was linked to the caps lock mechanism.
The photo below is that metal lever that sticks out on the top left corner of the typewriter.
When you nudge it in, the metallic link pushes all the way through the typewriter and disengages the caps lock, just like in the video below.
I guess the primary reason for such function is to prevent typists putting on a case and travelling with the carriage lifted i.e. in caps lock, which may lead to damage.