- \n
- RealSense\u2122 Depth Camera D415, D435, D435i or L515<\/li>\n\n\n\n
- Microsoft* Surface tablet, Windows* laptop or compatible Android* device<\/li>\n\n\n\n
- Dot3D Pro*<\/li>\n\n\n\n
- Meshlab*<\/li>\n\n\n\n
- Blender*<\/li>\n\n\n\n
- Firefox*<\/li>\n\n\n\n
- Optional \u2013 VR headset\u00a0<\/strong><\/li>\n<\/ul>\n\n\n\n
Step One: Creating the scan<\/strong><\/h3>\n\n\n\n
There are a number of 3D scanning applications out there that work with RealSense Depth cameras, but in this case we\u2019re choosing to use Dot3D Pro, because it\u2019s very easy to use, and allows us to quickly create a good scan and export the file. For the purposes of this demo, we used a Microsoft Surface Pro 4 (gen 6), but any device compatible with Dot3D Pro will work. Using a tablet makes the process of scanning easier, but a laptop or even desktop with a long enough USB cable could also be used.<\/p>\n\n\n\n
Once the camera is connected to the tablet (for this application, you must use a USB 3 cable and port), we open the software and select it within the settings window. From there, we can choose to create a new scan, and then simply select the \u201cscan\u201d icon to begin.<\/p>\n\n\n\n
![\"Creating](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/Annotation-2020-05-27-201007-1.jpg\")
<\/figure>\n\n\n\n<\/p>\n\n\n\n
As you move around, the pixels from the camera feed will turn white, yellow and green. Green means that that area is fully scanned. For a fully water tight scan, you should spend more time moving around the small detail areas and filling in any holes. When choosing a location, you can choose an outdoor space, but try and avoid extremely bright sunlight as your resulting model may look inconsistent or you may have some issues accurately scanning the space. If you are using the L515 for your scan, you should only scan indoor spaces for best results. As you can see from our scan, there were some issues with a shiny black table \u2013 this sort of material is not easy to capture with a depth camera, so whenever possible avoid anything that is very dark and reflective if you can.<\/p>\n\n\n\n
![\"Scanning](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/3.jpg\")
<\/figure>\n\n\n\n<\/p>\n\n\n\n
As you are moving through the space and doing as comprehensive scan as possible, you can also use the camera on your host device to take a high res RGB still that will be tied to the location you scanned it from \u2013 this can be used as later reference if you plan on working on your model in a 3D package like Blender or Maya.<\/p>\n\n\n\n
Once you are happy with the scan, tap the scan button again to finish. From here, you should optimize the model. This may take a few minutes depending on the size of your scan.<\/p>\n\n\n\n
Once optimized, save the scan as a .ply file.<\/p>\n\n\n\n
Step 2: Converting to OBJ.<\/strong><\/h3>\n\n\n\n
PLY files are a point cloud file, and can be used as is, for example, by uploading your PLY to sketchfab you can share the scan or view it in VR. However, if you want a bit more flexibility as to what you do with the file since PLY files are quite large, you may wish to convert it to a mesh. For this part of the process we are going to use Meshlab to convert the PLY to an OBJ.<\/p>\n\n\n\n
Open Meshlab, and go to File> Import Mesh. Import the PLY file you exported in the last step.<\/p>\n\n\n\n
![\"Meshlab](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/img1.jpg\")
Meshlab showing imported mesh.<\/figcaption><\/figure>\n\n\n\n From here, we\u2019re going to run a few operations to clean up the point cloud and turn it into a mesh. Depending on what your file looks like, you may want to spend some time playing around with the settings and deleting any extraneous vertices you didn\u2019t want. Use the \u201cSelect Vertices\u201d button on the top of the toolbar, and then select groups of vertices to delete using the \u201cDelete the current set of selected vertices\u201d tool.<\/p>\n\n\n\n
![\"Cleaned](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/img2.jpg\")
Cleaned up mesh with the two tools highlighted.<\/figcaption><\/figure>\n\n\n\n Next, go to Filters> Sampling > Poisson disk sampling ><\/strong>
\nOn the settings, make sure you have \u201cBase Mesh Subsampling\u201d selected, and change the number of samples to tens of thousands \u2013 I chose 35,000. The higher the number here, the more refined your final mesh will be. Try not to go too high though, the number of triangles will affect how your final mesh operates in other programs and applications.
\nIn the layer menu to the right, you should now see your original point cloud and the poisson sample. Delete the original mesh \u2013 we don\u2019t need it anymore.<\/p>\n\n\n\nFor the next step, go to Filters> Point set > Compute normal for point set<\/strong>. Change the neighbor number to 16 and run. This is trying to automatically determine which way each face will point.<\/p>\n\n\n\n
Now choose Filters> Remeshing, Simplification and Reconstruction> Surface Reconstruction: Ball Pivoting<\/strong>. Click the up arrow on the world unit box by the \u201cPivoting Ball Radius\u201d once \u2013 it should autofill an appropriate value. Apply, and we should now have a mesh instead of a point cloud. If you don\u2019t like the resulting mesh, you can go back and repeat these steps with slightly different parameters.<\/p>\n\n\n\n
So that we can take the color information we have along with us when we export from Meshlab, do the following steps.<\/p>\n\n\n\n
Run Filters>Texture>Parametrization: Trivial Per triangle<\/strong>. If you get an error, change the Inter-Triangle border value to 1. Next,<\/p>\n\n\n\n
Run Filters>Texture>Transfer Vertex color to texture<\/strong>. You will be asked to save the project at this stage, do so. Use the suggested name to save the texture file \u2013 it will have _tex.png appended to your project name.<\/p>\n\n\n\n
Export as an OBJ to the same folder. Make sure all the available check boxes are selected, and you will see the texture file you just created in the box to the right.<\/p>\n\n\n\n
This file type can also be used in 3D packages and game engines such as Unity or Unreal. In this case, we\u2019re going to move from Meshlab into Blender, an open source 3D editor.<\/p>\n\n\n\n
Step 3: Converting to GLB<\/strong><\/h3>\n\n\n\n
Open Blender, import your OBJ file. If you have trouble seeing anything when the file has been imported, click on \u201cView\u201d in the top left corner of the viewport and check the \u201cSidebar\u201d box. On the right side of the viewport, there\u2019s a tab labeled \u201cView\u201d. Change the clip start and end parameters to 0.01m and 10000m. Zoom in and out until you see your model. It may be upside down, so we\u2019ll need to rotate it properly and scale it down a bit.<\/p>\n\n\n\n
![\"Converting](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/image-3.jpg\")
The model in blender \u2013 highlighted on the left, the \u201crotate\u201d tool, and on the right, the \u201cView Panel\u201d is visible.<\/figcaption><\/figure>\n\n\n\n Click on the model. On the left side of the scene, select the \u201crotate\u201d icon, and then use the different orientation rings until you\u2019re happy that the floor is in the right orientation. Scale the model down at this stage too \u2013 we can tweak the final size in our next stage, but it\u2019s helpful to make it perhaps 10% of the size now.<\/p>\n\n\n\n
You may also notice that your model is missing it\u2019s texture. In the top right of the viewport window, there are some viewport shading icons. Choose the \u201cMaterial preview\u201d and you should see the colors for your model show up.<\/p>\n\n\n\n
You can also spend some time using Blender\u2019s editing tools to make your mesh look prettier \u2013 fill in holes, delete extraneous faces. There are many tutorials out there that can help you do that, so we are not going to go into it here. Once you\u2019re happy with the mesh, Export it as a \u201cGLTF 2.0\u201d. The file extension you actually need is the binary version of gltf, .glb.<\/p>\n\n\n\n
Step 4: Importing into Mozilla Hubs<\/strong><\/h3>\n\n\n\n
Mozilla hubs is a free platform that will allow you to bring other people into your space. We\u2019re going to create a brand new room, import our model, and then you will be able to share it with other people.<\/p>\n\n\n\n
Open Firefox, and go to hubs.mozilla.com. You\u2019ll need to sign up for an account. Click on \u201cCreate a room\u201d and then \u201cChoose Scene\u201d. Pick \u201cCreate a scene with Spoke\u201d<\/p>\n\n\n\n
Create a new project, and choose \u201cNew Empty Project\u201d.<\/p>\n\n\n\n
You should see an avatar representing your spawn point, and a crater terrain. In the bottom left panel, click on \u201cMy assets\u201d. Upload your .glb file here, and then drag it into the scene by dropping into the hierarchy panel just above the crater terrain. Use the spawn point icon and the crater terrain as a guide, to scale your scene appropriately \u2013 we scaled our mesh to 0.002 from the size it came in at. You can keep or hide the crater terrain, add any additional objects like lighting.<\/p>\n\n\n\n
![\"Importing](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/img4.jpg\")
Spoke window showing our final scene in the hierarchy and the viewport, scaled to fit the spawn point model.<\/figcaption><\/figure>\n\n\n\n When you\u2019re happy, select \u201cPublish to hubs\u201d. It\u2019s likely that you have too many polygons \u2013 our mesh is quite detailed, so it may not perform well on mobile. Ideally in the blender stage, we would decimate our mesh to below 50,000 polygons to make it perform well. All the other performance parameters should be fine. Publish your scene, select \u201cview your scene\u201d and then \u201cCreate a room with this scene.\u201d
\nOnce you enter the room, you can share the link with others and show them your scan. This workflow should work for any point cloud object you scan using Dot3D pro or other RealSense enabled software. Congratulations! You made a space that people can enter via a browser to chat with you, or even using a VR headset.
\nHere\u2019s the final scene. Obviously, this could use some work to further optimize and improve the mesh we created from our original high quality scan, but with some time and effort, you can use this workflow to create something great.<\/p>\n\n\n\n![\"The](\"https:\/\/realsense-redesign.fwc-staging.com\/wp-content\/uploads\/2020\/05\/img5.jpg\")
The final 3D scanned environment within the shareable, social Mozilla Hubs portal.<\/figcaption><\/figure>\n\n\n\n