The a similar phenomenon can be seen if you search for images of "spring graph layout", which show you many examples of such arcs, such as this one from wikipedia. Near the top on the right edge of the image, we see one such arc. Admittedly this isn't the best example. The top right corner of this image from this paper shows the effect a bit
Most of these graph visualizations are generated by simulating a spring force between each pair of connected nodes, and allowing the nodes to move according to this force.
In t-SNE, a similar interpretation of the algorithm is possible -- points in 2D space have a spring, whose resting length depends on the distance of the points in the original high-dimensional space. So points that are closer in 2D space than they are in high-dimensional space are pushed farther apart, and points that are farther in 2D space than in high-dimensional space are pulled together.
So it's likely that the arcs form because they are trying to maintain a constant distance to another group of points in the data.
Unlike in the spring graph layout above, every pair of points in t-SNE has a spring/force attached to it, so it's a valid question to ask why the arcs don't clump together into blobs, as shown in the graph visualization from wikipedia, where some groups of nodes on the edges have formed roundish clusters rather than arcs.
I suspect the reason for this is that each point in t-SNE has a variance attached to it. Points in a sparser region of the high-dimensional space have a higher variance compared to points in a lower dimensional space. The force on the springs of high-variance points is reduced, so if points in an arc were located in a sparse region of the original space, there would only be a weak force trying to pull them into a cluster, which might not overcome other opposing forces.
Furthermore, the method that the authors used to reduce crowding was to use a heavy-tailed distribution in the 2D space, which means points aren't penalized too heavily for being farther away from each other than they should. This also reduces the forces which would try to pull an arc into a cluster.