Originally the project description said to expand the functionality of TrondheimSim, a concept testing environment based on Statoils GeoSim. In the description the idea about video with GPS tracking is mentioned as a possible expansion.
Before the first meeting this idea was already being explored and taught about, and not just by me. This was clear when we sat down for the first time and everybody had multiple solutions and end user functionality already taught out. This gave me a lot of inspiration and drive to do good by Statoil and give it my all.
A couple of months before our meeting, a group of six students at my school had worked on, basically, the same project and had struggled with building the TrondheimSim environment. This resulted in no end user functionality expansion. With this in mind I set out to explore this environment problem and found out that there was no one at my school with the knowhow of setting up TrondheimSim. And with the response from Kongsberg Sim, who developed GeoSim for Statoil, on the question “how to build it”, turned out to being “dont try to build it”, me and my thesis supervisor, Tomas Holt, decided to use Google Earth as the main 3D environment for this project.
The decision to use Google Earth sat well with Statoils employment supervisor, Dr. Bjørn Magnus Sæter, because the main goal of this project was changing focus. With the focus point settling on testing out the possibilities with GPS video and GPS video cameras.
To sum up the beginning. This project turned out to be concept testing project which would explore the possibilities of GPS video and GPS video cameras. It would have a agile development process with researching, development and testing being a continuous cycle. Google Earth would be the main 3D environment where the video tracking would be visualized. And to top it all of, video capture would take place on a drone platform, helicopter or fixed wing
When the preliminary study was finished, I had landed on the decision of using my “native” programming language, Java, as main language for the application I wanted to develop. My plan was to have one application that could import videos, play them, extract the GPS data and send them to Google Earth. A bonus feature that would make this application perfect was that you could control the 3D environment with the application itself. Basically you would have two views, video in one, 3D tracking in the other and both controlled by the same control panel.
With the application planned out I could start the development, but first there were some technical issues that needed to be handled. The video camera uses the MP4 video container, the same as Apple products uses. In principle, everything is saved in its own track in the container. Video in track one, audio in track two and subtext in track three and so on. GPS data that is logged get stored as text in one of the subtext tracks. This meant that a subtext extractor needed to be made, and making it from scratch would take up too much time. So, many thanks to my technical consultant, Christian Aune Thomassen - CTO of WTW software, whom managed to find a useful API that could extract the GPS data from the MP4 video files. There were some good API`s out there but almost all of them were open source licensed, which meant that I could not use them.
So with the extraction out of the way and proven possible in Java, the next step was to design the GUI and create some functionality to get a look and feel of the application. Simple and sturdy, with that I mean that you get what you need and it works, no “gold plating”. Initialize, import and play functions were added.
With the initialize and import functions a question on storage was raised, and what would be the best way of storing all the information. To keep the application light weight and still have a secure and data safe storage I chose to use file storage on everything. This could be done with a common url file that links all the files that you need to get a hold of when you start the application. So the application only reads one file on startup and reads the other files only if necessary. This also keeps the IO traffic to a minimum.
In the spirit of light weight, all the media objects types like media and media player only gets created if a video is selected from the list. Then you also get to see the first frame get loaded in to view, and you can play the movie.
Its in the import function where all the magic really happens. The nmea GPS data extractor gets runned, and a .txt file is created and filled with the NMEA 0183 sentences. The video url and the nmea url gets added to the common url file, connecting the two. At the same time the NMEA sentences gets parsed and a GPS data model is created and later used in the plt generator and the kml generator, which both will be presented below.
The plt generator runs just after the GPS extractor, it uses the newly created data model to generate a .plt file. A .plt file, in this case, is used to describe a track with multiple points, each with their own GPS coordinates, altitude and time stamp. This can later be loaded into one of Statoils GeoSims depending on the location of the track. As seen in the result section tab.
The kml generator works a lot like the plt generator but the output is in another format. Kml is used by Google Earth, in this case, to describe a track. Google Earth loads the kml file and draws the the information, here a line between every data point from start to finish. Also as seen in the result section tab.
The application is linked to Google Earth and can initialize it. If a video, in the selection list, has focus when Google Earth is initialized it will automatically load the kml file related to the movie that has focus. It will draw the track and the track is now ready to get played.
To sum up the middle. You can with the application import a video and play it, the application will at the same time create description, .plt .kml, files for the 3D simulators, here GeoSim and Google Earth. It will initialize Google Earth and automatically load the kml file related to the video file that has focus, if you tell it to. All data is safely stored in separate files that all uses same “first name” and can easily be found in the runtime folder.