Zeiss has announced the release of its CinCraft Scenario which facilitates the tracking of optics for live, VFX, or LED studios, without the need for markers or external camera systems.
Vincent Tartar, UCO: What problem did you address, and what was the genesis of this system?
JB : I look at Zeiss as a solution provider. We know you face so many concerns when you are prepping for a film that will require VFX, and CinCraft Scenario is our way to address the problem. Overall, it over simplifies the post-production and even the pre-production process, especially when you see how simple the user interface is made.
SR : The Ncam company started in 2012, the first I saw was the Mark 1 system. It was pretty stunning to see everything this system could do, and the one thing that kept coming up was that the system was dependent on lens data. By then, Zeiss had already moved to XD lens data.
JB : In 2020 the Mark 2 system was released, so Sundeep approached them, and we started to talk to Ncam. We had something they needed, the lens data, then our two companies started to work together to develop a new system.
SR : The system had simplified over the years, but having the data directly from the manufacturer was gold. So we thought we could harness the best of both worlds to have a synergy, and find a way to stop having this time-consuming calibration process.
JB : Through time, synergy developed to the point that we said : let’s even discuss an acquisition, let’s have a better future than just being compatible. What if we could improve the issues people are facing on set today, what if we could transform this rather complex user interface into something way easier to use. ? Ncam had a pretty successful ten years in the industry, they were renowned in the broadcast industry, especially in sport programs. For it is important to mention, and that’s key, that this is the only system that can track outdoor.
SR : When we were looking at the technology, quite deeply, we were looking at other technologies as well, but they all had tracking points. Only Ncam system allowed us to go outside and track a scene without hassle, the setting up was quite simple, and it was obvious it was going to change the way people track scenes, especially in filmmaking.
By then in film production, Ncam was used on very high-end visual effects productions, but not widely in the feature film industry. There still were improvements to make it user-friendly, there was this extra distance we had to go to have this system to be used on a film set. A film set is a very active place, very creative and dynamic, it is constantly changing, you have to adapt to the ways shots are made, lights are changing, set is moving, and nobody knows what will come next. You don’t have this level of disruption in broadcast.
So that’s when we started to say, we have to create a system that will work with this kind of environment, and for this kind of operators also. Because you don’t have time to make that many changes on a film set. So we made a workshop, walking through the modus operandi on film set, how does it all work, and how can we adapt to that? There was an education for us, even though we knew how film set worked, we came up with a generic workflow to help us understand the needs on film sets.
Furthermore, we discovered cultural differences: there are different expectations on film sets, from one country to another. What happens in LA is different from what happens in NY, what happens in Europe is very different from how films are made in India, so we had to pull in all these different scenarios, to conceive the system.
We broke it down to the point that we defined five pillars that would help us conceive how the system should work : how the software should work, how you set the system up, where will you do the set-up, what to do when you arrive on the film set, and how does the lens calibration work? These are the things we discussed at length.
It was obvious that the user experience had to change, the system needed to be easy to learn and to use. Most of the tracking systems are very technical to use, and we wanted to answer this question : how a technical system could help solve a problem creatively? Every set is going to have a unique set of challenges, how do we solve them in a creative way? It was really where we wanted to go.
JB : Another ambition, in this process of over simplifying its use, is to create a standard. Let’s imagine a crew going on a three day shoot, choosing this camera, these lenses, all the gears needed, and then why not taking a tracking solution that you could find in any rental company? You would expect the rental company to prep it the same way they prep the other equipment. The other tracking systems work well, but this is the only system that will easily track indoor and outdoor, and that’s a real advantage when you’re shooting on location.
We come from a cine background, we make cine lenses, we address to the rental houses worldwide, they are our customers. So why not address this particular network, they are legitimate to hand over this system to their crew. It was one of the requirements while conceiving the CinCraft Scenario.
Actually, this system is the result of something we started to do a few years back. Around 2016/2017, when Zeiss released the eXtended Data protocol, very few people understood what was the purpose of these lens data, but we knew that we were coming to the age of digital filmmaking. Then we created a whole ecosystem named CinCraft. A couple of years ago, we released mapper, which is exactly about delivering specific information to VFX technicians and artists in a click, without having to eyeball the lens vignetting, for instance. And now we have CinCraft scenario which provides onset tracking but also onset recording of this metadata that we can then send to the post-production team.
So we are addressing two particular key VFX people issues, those doing match moving, which is very time-consuming and prone to errors, and those doing compositing who can easily add optical characteristics to their images.
SR : Another thing is that before we could live track, things were done backwards in a way. For instance, with match moving, first we were shooting then trying to find a way to adapt VFX to this moving image because the technology wasn’t there. Now, with new technologies, such as Unreal Engine, and tracking systems, it is possible to provide all these data. It’s a whole new paradigm.
JB : If we go a little into the details, we presented the system to many VFX supervisors, and the two immediate things that emerged were that there is no more calibration time and that we can shoot outdoor. But now what about the other lenses? It’s not just about Zeiss.
We are going to provide a catalog of other cine and broadcast lenses, only to let the user fine-tune the calibration, as we try to make the user save time. Until maybe third-party lens manufacturers will want to embark because in the end, if this system is seen by the industry as a standard, it would be even better for third-party lenses to be compatible and be “plug-n-play”. Since even if we are doing most of the work by providing data, the user will need to fine-tune the calibration when using third-party manufacturers lenses. We will also provide a calibration manual because there will always be off template lenses.
SR : Actually, one of the questions is how do we make this system an industry standard? The way to do this, you have to make this system scalable enough for people to adapt and learn. Ncam system and all the other fantastic tracking systems out there are pretty complex and complicated for people to learn. The number of things you must learn before you can use them can be daunting, and this is a barrier we would rather not have.
If we talk from a cinematographer perspective, we need them to be able to understand what is happening on set while looking at the system. We don’t want them to look at a tracking technician, having no idea what this technician is doing to make this work. Instead, we want them to be able to understand the process, where the data goes from, where they will be recorded to, just by looking at the tracking system. It should be that simple to understand. Without that scalability for the people, it can be a barrier to learn, this is why we made it simple to use and understand. I believe this is how you make it a standard that can be adopted.
JB : We’ve simplified it to the point it has a self-explanatory approach. We will put online a portal with tutorials and information on how to use the system. So anyone who is eager to learn how to operate it would find the answers there. We also have wizards in the user interface that will help the new user, or someone who hasn’t used it in a while, to be on the right tracks to use the system. There are people on set, like DITs, who could easily learn to set this up. To us, this is really the key to open up to the greater number of people.
It really comes down to budget allocations, you spend some money to rent the system, but it will save you time and cost later. And overall, this is for a greater quality of result. The VFX companies we presented the system to were seeing it as a relief to allow more creative work. They see it as a system that will provide accurate technical and time-saving information about things they don’t really like to do, but have to do. So the whole purpose it to help focus on a more creative work.
Technical
SR : When you look at the hardware, there is the Camera Bar with dual lenses. It has quite a wide field of view that does detect markers in the environment it’s seeing. Then there is a hardware called The Link, which is quite small to reduce the footprint and the weight of what’s on the camera. It acts has a hub, collecting data from the Camera Bar but also metadata from the camera and lens. The lens data can be collected in various ways, through SDI, Lemo, XD or even with lens motors which have rotary encoders if this is not a smart lens. And if the motors don’t provide an encoder, the system can provide a rotary encoder that can be daisy-chained to get the metadata from those non-smart lenses.
Then the data is passed on to a processing system, a little box we call The Origin, where all the data from the camera bar, but also from the lens and the camera, are aggregated and processed, to create a virtual lens model built from this data. This model will be passed on to a live rendering system, such as the Unreal Engine, for instance. The Origin is also recording on time code all the metadata it receives, especially from the lenses. Those recorded data can then be exported and used in the match moving process, making it simpler with the tracking data. It won’t eliminate the match moving process, but it will really accelerate that process. You can also use it for dailies.
JB : Another interesting feature is you can access the user interface without having to be wired to the system.
SR : It is a web-based interface where you can see what the Camera Bar sees, access data, and operate from there. It’s been optimized for mobile devices.
JB : To summarize this tracking system: It can track without markers, relying on imperfections or natural marks you’ll find where you’ll be filming. But it can also track reflective markers. There are situations where you can’t track natural markers because there aren’t any, like on green screens or blind surfaces. Then there is a new technology from Brompton, which is hidden markers that you can find on LED panels, and our system can track these invisible markers. So it’s really a three-in-one system, which makes it very adjustable to the filming situation you find yourself in.
Conclusion
The CinCraft Scenario offers a solution that appears easier to implement than some other systems, which may require more extensive logistics.
However, the CinCraft Scenario does retain some limitations, which we discussed with Jacques and Sundeep. Firstly, it’s a wired system that cannot rely on wireless technology due to the high risk of data loss. Moreover, in intense action scenes, for instance, the system may not always be able to maintain tracking. Lastly, it’s important to remember that the system relies on data provided by manufacturers, and it remains to be seen if other lens manufacturers will cooperate and supply their optical data to the CinCraft system to best meet users’ needs.
Thus, this system opens up intriguing possibilities; the advantage of not having to add tracking markers in the environment is a clear benefit, especially for smaller-budget productions or outdoor shoots. The next step will be to test the CinCraft Scenario ourselves, which will be the focus of an upcoming article where we’ll see it in real-world conditions and provide practical feedback on this highly promising system.
