Take a look at this video:
It shows a ball following a curved path because of magnus effect. This is a 2D video. Every frame of this video is a projection of the objects in front of the camera on a 2D plane (the film of the camera).
Is it possible to extract the positions of the ball in 3d space from this video?
It certainly is. That’s exactly what you’re doing while you’re watching this video. You can intuitively infer that the ball is going away from the camera.
How do we do that?
We observe that the ball is diminishing in size. This tells us that the distance of ball from us is increasing. This fact gives us a way to deduce the depth of the ball in this video. Initially the ball is about 1m away (even if we get this wrong, the shape of the path in 3D space won’t be distorted, it’ll just be scaled by the factor of our error). If the size of the projection is one-third in a later frame, we can deduce that the ball is 3m away from camera at the instance of that frame. That’s one of the three coordinates we need to plot the path of the ball.
Extracting Data From The Video
First of all we extract the 2D coordinates of the image of the ball in frame. I made a custom tool for this task:
It allows you to mark the position of the ball for the frames of the video. If the marker doesn’t perfectly fit the ball image, you can specify one of the 8 directions for sub pixel interpolation, to account for the offset between the ball and the marker. I collected two data points for each frame. One with the marker just bigger than the ball image, and the other with marker just smaller than the ball image, and used the average of those two values for further calculations.
This tool gave me a list of the ball image position and size for each frame.
How do we get the rest of the coordinates?
Assume a cartesian space with origin at camera, x-axis going to the right, z-axis in the direction of camera, and y-axis deduced relative to previous two (almost vertical, since camera is pointing just above the horizon). Imagine a plane parallel to the y-z plane going through the ball. The distance from this plane from the y-z plane is the x coordinate of the ball.
The ratio of this distance to the size of the actual ball will equal the ratio of length of black segment (in the above picture) to the size of the image of the ball (the image is similar to the plane going through the ball and parallel to x-y plane). Here’s our x-coordinate. We can find y-coordinate similarly. z-coordinate can be calculated using pythagorean theorem.
Visualization of the data
I used blender for this task. Here’s the result:
As you can see, the curvature is very mild.
- I had to skip the first couple of frames since the foot was blocking the view.
- Another thing that I had in mind was calculating the velocity, acceleration and visualizing the components of the force on the ball, but the error was too high (inversely proportional to size of the image of ball) to extract any meaningful quantitative data.
- The z coordinate of the ball near the end was around 33m, which makes sense wrt the size of a soccer field.
Let me know if you would like to see an article going into the implementation details
See you later.