I've been to New York City a few months ago. It was an interesting experience. Everything is so big, it is difficult to get a feeling for it if you haven't seen it yourself. For example, the WTC Memorial is overwhelmingly large. I tried to capture that in a photograph with my Panasonic DMC-TZ41, but it just doesn't look that large:
A photosphere captures that much better (it looks great on my Smartphone, but not nearly as well on the web): Android Photosphere
Optonaut goes one step further. This app is able to create 3D photospheres.
How it works - User perspective
You start the app, slowly turn around yourself to capture everything around you. This includes your feet and the sky, if you really want a complete photosphere. The recording process the same as for Android Photospheres, but probably with a slightly different interface:
When you want to watch it in 3D you have to have something that holds it in front of your face at the right distance. Cardboard is an easy and cheap possibility to do so:
And, of course, you can already share optographs:
How it works - Developers perspective
Optonaut makes use of the camera and takes a lot of images. Those images are combined to a photosphere by a process called image stitching. This means you search points which seem to match to build one big image from many small images. A big problem is that your camera moves quite a lot in space and the environment changes. So there might not be any way to make a perfect match. Sensors like the compass and the accelerometer help to figure out at which direction you are currently pointing. Programs like hugin stitch images semi-automatically, but Optonaut works completely automatically.
From my experience with Android Photospheres, this works much better when the objects around you are far away. That means it doesn't work in your room.
Then Optonaut uses a technique called stereoscopy. They show you different images for each eye to give you the impression of a 3D image. They also use a different photosphere techique to "mix images dynamically". Optonaut claims that this does work for close objects (in contrast to the traditional photosphere apps like Google Photosphere, Photosynth or Microsoft ICE).
As they can figure out where you look at with the accelerometer / compass, they adjust the images accordingly for each eye to create a 3D impression:
Now it gets very interesting. Usually, you would need two cameras next to each other to take two slightly shifted images. But Optonaut needs only one camera. They make use of the fact that you rotate around your axis to photograph your environment. Optonaut takes two images which are next to each other, match them, deskew them. Et voilà: You are able to get a 3D impression!
A side note: The 3D photosphere is stored as an archive of images.
The Optonaut Team
Optonaut has two founders: Emanuel Jöbstl and Johannes Schickling. I've had the luck to work with them in my first hackathon (CODEFEST8). We won in Karlsruhe with an App / web service with a fully automatic drivers logbook. Both of them are great developers. Emanuel created the (working!) Android app which connected to the car within a couple of hours and Johannes made a beatiful frontend in the same time.
Emanuel knows a lot about protocols, can understand mathematically complex algorithms and solve conceptually difficult problems. I guess he wrote the stitching algorithm.
I've learned a lot from Johannes about tools. He is a perfectionist. In situations where most people would say "hey, it works, lets drink some coffee" he is still working hard to get the best user experience, making the code more maintainable by refactoring it, improving the way how code is built / deployed. In a short-term project this costs probably more time than doing it quick-and-dirty, but on the medium/long run (e.g. more than 2 weeks) it saves a lot of work.
And Optonaut is such a long-term project. They are working on it for quite a while now, so I am sure the quality is pretty high. I am very curious if the Kickstarter campaign works as well as they hope and I cannot wait for the first people to share their optographs from amazing places!
Future of Optonaut
The Optonaut team plans to build a sharing site where you can show others amazing places. They want to completely redesign the App.
I guess the stitching algorithm is something where you can put endless effort in. It is an optimization problem where it is not completely clear what to optimize, so I guess this will continually be improved.
I want it! What do I need?
You need an Android / iOS smartphone. There seems to be a web-based viewer which works with Chrome/Firefox/Safari, too.
See also
- Optonaut.com: The official website
- demo.optonaut.com: In case you have a smartphone and a Cardboard, you can try the web demo already.
- Kickstarter: Support them, get early access / the App and your own Cardboard