For some background into what this series is all about, see this previous post.
I’ve been tracking Photosynth for some time, but only recently became aware of its use of point-clouds on the back-end and the possibility of extracting this information from the site. I first got inspired by Binary Millenium’s video of the process they’ve used along with the Python script they’ve provided on their website to extract the points from a Photosynth point cloud file. I converted this code to C# (without realising there was already a version out there – I should really have done better research, but anyway), and then eventually came across a much more complete exporter on CodePlex which filled in some of the gaps in the original script (such as how to properly skip the file headers).
The process that Binary Millenium outlined makes use of a network monitor tool such as Wireshark to detect point cloud files accessed on the Photosynth server. The Photosynth client is Silverlight based, and accesses a set of files on a Photosynth server, each of which stores up to 5,000 points. These files usually take the form of a file named “points_0_0.bin” (where the two zeroes are incremented, although it’s not clear to me the logic driving the change in these indeces, as yet).
So the approach I’ve used is to incrementally download and process the files in a particular set, first by incrementing the minor number (the second) and then – should there be no more “minor” files – the major number. And if there isn’t another “major” file in the sequence, then we stop.
[It should be noted that the technique we’re using to extract the points from Photosynth is really a backdoor, although there seems to be a fair amount of interest in an official export mechanism.]
Processing is reasonably straightforward: we chunk through each file, adding each of its points to a text file. We do this because AutoCAD currently doesn’t provide a point cloud creation API, so we create a text file and convert it to the primary supported format (.LAS) using the freely available txt2las tool (which you need to place in the same folder as the application’s DLL for this to work).
We then index the point cloud using the POINTCLOUDINDEX command, and then bring it into the current editing session using the POINTCLOUDATTACH command. So no, we’re not actually using an API to create or manipulate point clouds inside AutoCAD (as yet), but this is the approach that makes the most sense, for now. And it works well.
Here’s the C# code:
using Autodesk.AutoCAD.ApplicationServices;
using Autodesk.AutoCAD.DatabaseServices;
using Autodesk.AutoCAD.EditorInput;
using Autodesk.AutoCAD.Runtime;
using Autodesk.AutoCAD.Geometry;
using Autodesk.AutoCAD.Colors;
using System.Diagnostics;
using System.Text.RegularExpressions;
using System.Reflection;
using System.IO;
using System.Net;
using System;
namespace ImportPhotosynth
{
public class Commands
{
[CommandMethod("IMPORTPHOTOSYNTH")]
public void ImportPhotosynth()
{
Document doc =
Application.DocumentManager.MdiActiveDocument;
Database db = doc.Database;
Editor ed = doc.Editor;
HostApplicationServices ha =
HostApplicationServices.Current;
PromptResult pr =
ed.GetString(
"Enter URL of first Photosynth point cloud: "
);
if (pr.Status != PromptStatus.OK)
return;
string path = pr.StringResult;
pr =
ed.GetString(
"Enter name of Photosynth point cloud: "
);
if (pr.Status != PromptStatus.OK)
return;
string name = pr.StringResult;
// The root path has "points_0_0.bin" on the end.
// Strip off the last 5 characters ("0_0.bin"), so
// that we can compose the sequence of URLs needed
// for each of the point cloud files (usually
// going up to about "points_0_23.bin")
if (path.Length > 5)
path = path.Substring(0, path.Length - 7);
// We'll store most local files in the temp folder.
// We get a temp filename, delete the file and
// use the name for our folder
string localPath = Path.GetTempFileName();
File.Delete(localPath);
Directory.CreateDirectory(localPath);
localPath += "\\";
// Paths for our temporary files
string txtPath = localPath + "points.txt";
string lasPath = localPath + "points.las";
// Our PCG file will be stored under My Documents
string outputPath =
Environment.GetFolderPath(
Environment.SpecialFolder.MyDocuments
) + "\\Photosynth Point Clouds\\";
if (!Directory.Exists(outputPath))
Directory.CreateDirectory(outputPath);
// We'll use the title as a base filename for the PCG,
// but will use an incremented integer to get an unused
// filename
int cnt = 0;
string pcgPath;
do
{
pcgPath =
outputPath + MakeValidFileName(name) +
(cnt == 0 ? "" : cnt.ToString()) + ".pcg";
cnt++;
}
while (File.Exists(pcgPath));
// The path to the txt2las tool will be the same as the
// executing assembly (our DLL)
string exePath =
Path.GetDirectoryName(
Assembly.GetExecutingAssembly().Location
) + "\\";
if (!File.Exists(exePath + "txt2las.exe"))
{
ed.WriteMessage(
"\nCould not find the txt2las tool: please make sure it " +
"is in the same folder as the application DLL."
);
return;
}
// Start our progress meter
ProgressMeter pm = new ProgressMeter();
using (pm)
{
pm.SetLimit(100);
pm.Start("Downloading/processing Photosynth points");
// Counters for the major and minor file numbers and
// a total count
int maj = 0, min = 0, totalFiles = 0;
bool cont = true;
bool first = true;
long totalPoints = 0;
// Create our intermediate text file in the temp folder
FileInfo t = new FileInfo(txtPath);
StreamWriter sw = t.CreateText();
using (sw)
{
pm.MeterProgress();
// We'll use a web client to download each .bin file
WebClient wc = new WebClient();
using (wc)
{
while (cont)
{
// Loop for each .bin file
string root =
maj.ToString() + "_" + min.ToString() + ".bin";
string src = path + root;
string loc = localPath + root;
try
{
wc.DownloadFile(src, loc);
}
catch
{
// First time we fail, reset min and increment maj
if (first)
{
min = 0;
maj++;
first = false;
}
else
{
// The second time there's an error, it means
// we can't get the next major version
cont = false;
}
}
if (File.Exists(loc))
{
// Reset the failure flag, as we have a valid file
first = true;
ed.WriteMessage("\npoints_" + root);
// Open our binary file for reading
BinaryReader br =
new BinaryReader(
File.Open(loc, FileMode.Open)
);
using (br)
{
try
{
// First information is the file version
// (for now we support version 1.0 only)
ushort majVer = ReadBigEndianShort(br);
ushort minVer = ReadBigEndianShort(br);
if (majVer != 1 || minVer != 0)
{
ed.WriteMessage(
"\nCannot read a Photosynth point cloud " +
"of this version ({0}.{1}).",
majVer, minVer
);
return;
}
pm.MeterProgress();
// Clear some header bytes we don't care about
int n = ReadCompressedInt(br);
for (int i = 0; i < n; i++)
{
int m = ReadCompressedInt(br);
for (int j = 0; j < m; j++)
{
ReadCompressedInt(br);
ReadCompressedInt(br);
}
}
// Find out the number of points in the file
int nPoints = ReadCompressedInt(br);
totalPoints += nPoints;
ed.WriteMessage(" - {0} points\n", nPoints);
// We want to tick the progress meter four times
// per file
int interval = nPoints / 4;
for (int i = 0; i < nPoints; i++)
{
if (interval > 0 && i % interval == 0)
pm.MeterProgress();
System.Windows.Forms.Application.DoEvents();
// Give the user the chance to escape
if (ha.UserBreak())
{
ed.WriteMessage(
"\nImport cancelled. Run the " +
"IMPORTPHOTOSYNTH command to try again."
);
pm.Stop();
sw.Close();
br.Close();
File.Delete(loc);
CleanupTmpFiles(txtPath);
return;
}
// Read our coordinates
float x = ReadBigEndianFloat(br);
float y = ReadBigEndianFloat(br);
float z = ReadBigEndianFloat(br);
// Read and extract our RGB values
UInt16 rgb = ReadBigEndianShort(br);
int r = (rgb >> 11) * 255 / 31;
int g = ((rgb >> 5) & 63) * 255 / 63;
int b = (rgb & 31) * 255 / 31;
// Write the point with its color to file
sw.WriteLine(
"{0},{1},{2},{3},{4},{5}", x, y, z, r, g, b
);
}
}
catch (System.Exception ex)
{
ed.WriteMessage(
"\nError processing point cloud file " +
"\"points_{0}\": {1}",
root, ex.Message
);
}
}
// Delete our local .bin file
File.Delete(loc);
// Increment our counters
min++;
totalFiles++;
}
}
ed.WriteMessage(
"\nImported {0} points from {1}" +
"point cloud files downloaded from Photosynth.\n" +
"Converting the text file to a .LAS.\n",
totalPoints, totalFiles
);
// Close the text file stream
sw.Close();
// Stop the progress meter
pm.Stop();
if (totalFiles > 0)
{
// Use the txt2las utility to create a .LAS
// file from our text file
ProcessStartInfo psi =
new ProcessStartInfo(
exePath + "txt2las",
"-i \"" + txtPath +
"\" -o \"" + lasPath +
"\" -parse xyzRGB"
);
psi.CreateNoWindow = false;
psi.WindowStyle = ProcessWindowStyle.Hidden;
// Wait up to 20 seconds for the process to exit
try
{
using (Process p = Process.Start(psi))
{
p.WaitForExit(20000);
}
}
catch
{ }
// If there's a problem, we return
if (!File.Exists(lasPath))
{
ed.WriteMessage(
"\nError creating LAS file."
);
return;
}
File.Delete(txtPath);
ed.WriteMessage(
"Indexing the LAS and attaching the PCG.\n"
);
// Index the .LAS file, creating a .PCG
string lasLisp = lasPath.Replace('\\', '/'),
pcgLisp = pcgPath.Replace('\\', '/');
doc.SendStringToExecute(
"(command \"_.POINTCLOUDINDEX\" \"" +
lasLisp + "\" \"" +
pcgLisp + "\")(princ) ",
false, false, false
);
// Attach the .PCG file
doc.SendStringToExecute(
"_.WAITFORFILE \"" +
pcgLisp + "\" \"" +
lasLisp + "\" " +
"(command \"_.-POINTCLOUDATTACH\" \"" +
pcgLisp +
"\" \"0,0\" \"1\" \"0\")(princ) ",
false, false, false
);
doc.SendStringToExecute(
"_.-VISUALSTYLES _C _Conceptual _.ZOOM _E ",
false, false, false
);
}
}
}
}
}
// A command which waits for a particular PCG file to exist
[CommandMethod("WAITFORFILE", CommandFlags.NoHistory)]
public void WaitForFileToExist()
{
Document doc =
Application.DocumentManager.MdiActiveDocument;
Database db = doc.Database;
Editor ed = doc.Editor;
HostApplicationServices ha =
HostApplicationServices.Current;
PromptResult pr = ed.GetString("Enter path to PCG: ");
if (pr.Status != PromptStatus.OK)
return;
string pcgPath = pr.StringResult.Replace('/', '\\');
pr = ed.GetString("Enter path to LAS: ");
if (pr.Status != PromptStatus.OK)
return;
string lasPath = pr.StringResult.Replace('/', '\\');
// Check the write time for the PCG file...
// if it hasn't been written to for at least one second,
// we can continue
TimeSpan ts = new TimeSpan(100);
do
{
if (File.Exists(pcgPath))
{
DateTime dt = File.GetLastWriteTime(pcgPath);
ts = DateTime.Now - dt;
}
else
{
ts = new TimeSpan(ts.Ticks + 100);
}
System.Windows.Forms.Application.DoEvents();
}
while (ts.Seconds < 1);
try
{
CleanupTmpFiles(lasPath);
}
catch
{ }
}
private void CleanupTmpFiles(string txtPath)
{
if (File.Exists(txtPath))
File.Delete(txtPath);
Directory.Delete(
Path.GetDirectoryName(txtPath)
);
}
private static int ReadCompressedInt(BinaryReader br)
{
int i = 0;
byte b;
do
{
b = br.ReadByte();
i = (i << 7) | (b & 127);
}
while (b < 128);
return i;
}
private static float ReadBigEndianFloat(BinaryReader br)
{
byte[] b = br.ReadBytes(4);
return BitConverter.ToSingle(
new byte[] { b[3], b[2], b[1], b[0] },
0
);
}
private static UInt16 ReadBigEndianShort(BinaryReader br)
{
byte b1 = br.ReadByte();
byte b2 = br.ReadByte();
return (ushort)(b2 | (b1 << 8));
}
private static string MakeValidFileName(string name)
{
string invChars =
Regex.Escape(new string(Path.GetInvalidFileNameChars()));
string invRegEx = string.Format(@"[{0}]", invChars + ".");
return Regex.Replace(name, invRegEx, "-");
}
}
}
To give this try, let’s run the code with this Photosynth of Pembroke Castle:
We can run the IMPORTPHOTOSYNTH command, passing in the initial point cloud URL as determined by Wireshark:
Command: IMPORTPHOTOSYNTH Enter URL of first Photosynth point cloud: http://mslabs-840.vo.llnwd.net/d2/photosynth/m6/collections/68/14/1b/68141b01-fd02-468e-934a-004769a0c30b.synth_files/points_0_0.bin Enter name of Photosynth point cloud: "Pembroke Castle"
As the code downloads and processes the various files, you should see appropriate output at the command-line. Now without downloading the files up front, there’s no way I know of to tell how many they’ll be, so the progress metre is often inaccurate (unless we happen to have exactly 20 files, which is a fairly typical number).
Here’s the output you’ll hopefully see:
points_0_0.bin - 5000 points
points_0_1.bin - 5000 points
points_0_2.bin - 5000 points
points_0_3.bin - 5000 points
points_0_4.bin - 5000 points
points_0_5.bin - 5000 points
points_0_6.bin - 5000 points
points_0_7.bin - 5000 points
points_0_8.bin - 5000 points
points_0_9.bin - 5000 points
points_0_10.bin - 5000 points
points_0_11.bin - 5000 points
points_0_12.bin - 5000 points
points_0_13.bin - 5000 points
points_0_14.bin - 5000 points
points_0_15.bin - 5000 points
points_0_16.bin - 5000 points
points_0_17.bin - 5000 points
points_0_18.bin - 5000 points
points_0_19.bin - 5000 points
points_0_20.bin - 5000 points
points_0_21.bin - 5000 points
points_0_22.bin - 1849 points
points_1_0.bin - 2456 points
Imported 114305 points from 24 point cloud files downloaded from Photosynth.
Converting the text file to a .LAS.
Indexing the LAS and attaching the PCG.
Command: _.POINTCLOUDINDEX
Path to data file to index:
C:/Users/walmslk/AppData/Local/Temp/tmpD280.tmp/points.las
Path to indexed point cloud file to create
<C:\Users\walmslk\AppData\Local\Temp\tmpD280.tmp\points.pcg>:
C:/Users/walmslk/Documents/Photosynth Point Clouds/Pembroke Castle.pcg
Converting C:\Users\walmslk\AppData\Local\Temp\tmpD280.tmp\points.las to
C:\Users\walmslk\Documents\Photosynth Point Clouds\Pembroke Castle.pcg in the background.
Command:
Command: Enter path to PCG: Enter path to LAS:
Command: _.-POINTCLOUDATTACH
Path to point cloud file to attach: C:/Users/walmslk/Documents/Photosynth Point
Clouds/Pembroke Castle.pcg
Specify insertion point <0,0>:0,0
Current insertion point: X = 0.0000, Y = 0.0000, Z = 0.0000
Specify scale factor <1>:1
Current scale factor: 1.000000
Specify rotation angle <0>:0
Current rotate angle: 0
1 point cloud attached
Command:
Command: Regenerating model.
Command:
Enter an option [set Current/Saveas/Rename/Delete/?]:
Enter an option
[2dwireframe/Wireframe/Hidden/Realistic/Conceptual/Shaded/shaded with
Edges/shades of Gray/SKetchy/X-ray/Other] <2dwireframe>:
And then there’s the model itself, of course:
Once completed, an appropriate PCG will have been created and placed in your “My Documents\Photosynth Point Clouds”. If you want to keep the intermediate text and .LAS files, then it should be a simple matter to comment out the code that erases them from the temporary files folder (I’ve done my best to have the application clean up extraneous files from the file system as it goes along).
One other thought I’ve had on this implementation… as the various point cloud files are independent, and could presumably be added in any order into our intermediate text files, this is a highly parallelizable bit of code, and would be perfect for F#’s Asynchronous Workflows. It would help to know exactly what files are available to us before downloading/processing them, but if this were possible it would be a very cool use of that particular mechanism.
So what’s next? Well, Wireshark is fine for testing, but it’s a bit cumbersome to be used as part of a consistently applied process. I really wanted to avoid having this manual step (although I admit I found it intriguing to take the lid off the HTTP traffic generated by a typical browsing session: I’m much more used to using tools such as Process Monitor to check Registry and/or file system access, but this was new for me).
In the next post in this series we’re going to make use of another freely available web browser component that provides us with HTTP event information, so we can provide a UI that builds a list of point clouds as the user browses through a hosted Photosynth session, for later importing into AutoCAD.
