Hello!
This is the first of two videos about drone photogrammetry of my series "Professional photogrammetry".
In this video I will explain everything that I know about scanning for visual effects using commercial drones.
We will talk about shooting patterns, surveying extra data, equipment, flight missions, etc. And we will also start processing some of the data sets that I have available for you.
The video is around 3 hours long and it will continue in the next episode of this series.
Head over my patreon for more information
Thanks!
Xuan.
Photoscan
Houdini topo transfer - aka wrap3 /
For a little while I have been using Houdini topo transfer tools instead of Wrap 3. Not saying that I can fully replace Wrap3 but for some common and easy tasks, like wrapping generic humans to scans for both modelling and texturing, I can definitely use Houdini now instead of Wrap 3.
Wrapping generic humans to scans
This technique will allow you to easily wrap a generic human to any actor’s scan to create digital doubles. This workflow can be used during modeling the digital double and also while texturing it. Commonly, a texture artist gets a digital double production model in t-pose or a similar pose that doesn’t necessary match the scan pose. It is a great idea to match both poses to easily transfer color details and surface details between the scan and the production model.
For both situations, modeling or texturing, this is a workflow that usually involves Wrap3 or other proprietary tools for Maya. Now it can also easily be done in Houdini.
First of all, open the ztool provided by the scanning vendor in Zbrush. These photogrammetry scans are usually something around 13 – 18 million of polygons. Too dense for the wrapping process. You can just decimate the model and export it as .obj
In Maya align roughly your generic human and the scan. If the pose is very different, use your generic rig to match (roughly) the pose of the scan. Also make sure both models have the same scale. Scaling issues can be fixed in Wrap3 or Houdini in this case, but I think it is better to fix it beforehand, in a vfx pipeline you will be publishing assets from Maya anyway. Then export both models as .obj
It is important to remove teeth, the interior of the mouth and other problematic parts from your generic human model. This is something you can do in Houdini as well, even after the wrapping, but again, better to do it beforehand.
Import the scan in Houdni.
Create a topo transfer node.
Connect the scan to the target input of the topo transfer.
Bring the base mesh and connect it to the source input of the topo transfer.
I had issues in the past using Maya units (decimeters) so better to scale by 0.1 just in case.
Enable the topo transfer, press enter to activate it. Now you can place landmarks on the base mesh.
Add a couple of landmarks, then ctrl+g to switch to the scan mesh, and align the same landmarks.
Repeat the process all around the body and click on solve.
Your generic human will be wrapped pretty much perfectly to the actor’s scan. Now you can continue with your traditional modeling pipeline, or in case you are using this technique for texturing, move into Zbrush, Mari and or Houdini for transferring textures and displacement maps. There are tutorials about these topics on this site.
Transferring texture data
Import the scan and the wrapped model into Houdini.
Assign a classic shader with the photogrammetry texture connected to its emission color to the scan. Disable the diffuse component.
Create a bakeTexture rop with the following settings.
Resolution = 4096 x 4096.
UV object = wrapped model.
High res object = scan.
Output picture = path_to_file.%(UDIM)d.exr
Format = EXR.
Surface emission color = On.
Baking tab = Tick off Disable lighting/emission and Add baking exports to shader layers.
If you get artifacts in the transferred textures, in the unwrapping tab change the unwrap method to trace closest surface. This is common with lidar, photogrammetry and other dirty geometry.
You can run the baking locally or on the farm.
Take a look at the generated textures.
On-set tips: Creating high frequency detail /
In a previous post I mentioned the importance of having high frequency details whilst scanning assets on-set. Sometimes if we don't have that detail we can just create it. Actually sometimes this is the only way to capture volumes and surfaces efficiently, specially if the asset doesn't have any surface detail, like white objects for example.
If we are dealing with assets that are being used on set but won't appear in the final edit, it is probably that those assets are not painted at all. There is no need to spend resources on it, right? But we might need to scan those assets to create a virtual asset that will be ultimately used on screen.
As mentioned before, if we don't have enough surface detail it will be so difficult to scan assets using photogrammetry so, we need to create high frequency detail on our own way.
Let's say we need to create a virtual assets of this physical mask. It is completely plain, white, we don't see much detail on its surface. We can create high frequency detail just painting some dots, or placing small stickers across the surface.
In this particular case I'm using a regular DSLR + multi zoom lens. A tripod, a support for the mask and some washable paint. I prefer to use small round stickers because they create less artifacts in the scan, but I run out of them.
I created this support while ago to scan fruits and other organic assets.
The first thing I usually do (if the object is white) is covering the whole object with neutral gray paint. It is way more easy to balance the exposure photographing again gray than white.
Once the gray paint is dry I just paint small dots or place the round stickers to create high frequency detail. The smallest the better.
Once the material has been processed you should get a pretty decent scan. Probably an impossible task without creating all the high frequency detail first.
On-set tips: The importance of high frequency detail /
Quick tip here. Whenever possible use some kind of high frequency detail to capture references for your assets. In this scenario I'm scanning with photos this huge rock, with only 50 images and very bad conditions. Low lighting situation, shot hand-held, no tripod at all, very windy and raining.
Thanks to all the great high frequency detail on the surface of this rock the output is quite good to use as modeling reference, even to extract highly detailed displacement maps.
Notice in the image below that I'm using only 50 pictures. Not much you might say. But thanks to all the tiny detail the photogrammetry software does very well reconstructing the point cloud to generate the 3D model. There is a lot of information to find common points between photos.
The shooting pattern couldn't be more simple. Just one eight all around the subject. The alignment was completely successfully in Photoscan.
As you can see here, even with a small number of photos and not the best lighting conditions, the output is quite good.
I did an automatic retopology in Zbrush. I don't care much about the topology, this asset is not going to be animated at all. I just need a manageable topology to create a nice uv mapping and reproject all the fine detail in Zbrush and use it later as displacement map.
A few render tests.
Photogrammetry tests /
I've been doing a lot of photogrammetry stuff recently, can't show much yet but I will soon.
These are just a few tests that I did to get comfortable scanning small props.
Meshlab align to ground /
If you deal a lot with 3D scans, Lidars, photogrammetry and other heavy models, you probably use Meshlab. This "little" software is great managing 75 million polygon Lidars and other complex meshes. Photoscan experienced users usually play with the align to ground tool to establish the correct axis for their resulting meshes.
If you look for this option in Meshlab you wouldn't find it, at least I didn't. Please let me know if you know how to do this.
What I found is a clever workaround to do the same same thing with a couple of clicks.
- Import your Lidar or photogrammetry, and also import a ground plane exported from Maya. This is going to be your floor, ground or base axis.
- This is a very simple example. The goal is to align the sneaker to the ground. I would like to deal with such a simple lidars at work :)
- Click on the align icon.
- In the align tool window, select the ground object and click on glue here mesh.
- Notice the star that appears before the name of the object indicating that the mesh has been selected as base.
- Select the lidar, photogrammetry or whatever geometry that need to be aligned and click on point based glueing.
- In this little windows you can see both objects. Feel free to navigate around it behaves like a normal viewport.
- Select one point at the base of the lidar by double clicking on top of it. Then do the same in one point of the base geo.
- Repeat the same process. You'll need at least 4 points.
- Done :)