The code to do this – from my side of things, at least – proved reasonably straightforward. To understand what was needed, let’s talk a little about Project Photofly’s underlying architecture…
On a side note, both Photofly and Photosynth try to minimise the time it takes to upload and process sets of images: they uniquely identify images during upload using a checksum of their contents and properties, so only new images need uploading/processing.
My hope when I first started looking at Photofly was that a well-defined, immediately-usable web service interface would be available to generate photo scenes using either SOAP or REST. That isn’t currently the case: client applications make low-level calls to various AWS services, to upload the images to S3 and then request them to be stitched into a scene, presumably using EC2. The short-term solution the Photofly team were able to provide was a simply command-line executable (PhotoflyClient.exe) built from source code extracted from the Photo Scene Editor codebase which takes care of the uploading and messaging to AWS. My application can then simply point this executable at the folder of images and wait for it to complete. The resultant RZI file – a simple XML format – can then be transformed into plain text using XSLT, which can be passed into the code used previously for importing from Photosynth (which uses the TXT2LAS tool to generate a LAS file followed by the POINTCLOUDINDEX and POINTCLOUDATTACH commands to bring the data into AutoCAD).
The current version of this PhotoflyClient executable was really only tended to prove the concept: there are some dependencies on libraries that probably shouldn’t be needed in a simple command-line executable (such as Qt), so we’re going to tidy that up before we actually post it. We also expect to develop a better, publicly accessible encapsulation of the Photofly service (probably via a SOAP-exposed web service, but that’s a detail we’ll get to in due course) at some point in the future. Such an API would allow more granular control – and information – on what is happening
But anyway – here’s the C# code that makes use of this executable to generate a point cloud from 2D photos in AutoCAD 2011.
using Autodesk.AutoCAD.ApplicationServices;
using Autodesk.AutoCAD.EditorInput;
using Autodesk.AutoCAD.Runtime;
using Autodesk.AutoCAD.Windows;
using System.Text.RegularExpressions;
using System.ServiceModel;
using System.Reflection;
using System.IO;
using System.Xml;
using System.Xml.Xsl;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System;
namespace ImportPhotofly
{
public class Commands
{
static FileSystemWatcher _fsw = null;
static TrayItem _ti = null;
static System.Drawing.Icon _procIcon = null;
static System.Drawing.Icon _readyIcon = null;
[DllImport("user32.dll")]
private static extern IntPtr SendMessageW(
IntPtr hWnd, int Msg, IntPtr wParam,
ref COPYDATASTRUCT lParam
);
[DllImport("user32.dll")]
private static extern bool InvalidateRect(
IntPtr hWnd, ref System.Drawing.Rectangle rect, bool bErase
);
// The structure we'll require for SendMessageW
private struct COPYDATASTRUCT
{
public IntPtr dwData;
public int cbData;
public IntPtr lpData;
}
[CommandMethod("UP")]
public void UploadToPhotofly()
{
Document doc =
Application.DocumentManager.MdiActiveDocument;
Editor ed = doc.Editor;
// Ask the user to browse to a folder of images
System.Windows.Forms.FolderBrowserDialog fdb =
new System.Windows.Forms.FolderBrowserDialog();
fdb.ShowNewFolderButton = false;
fdb.RootFolder = Environment.SpecialFolder.MyComputer;
fdb.Description =
"Select folder of images from which to " +
"generate point cloud.";
System.Windows.Forms.DialogResult dr = fdb.ShowDialog();
if (dr != System.Windows.Forms.DialogResult.OK)
return;
string imgPath = fdb.SelectedPath;
// Get the name used for the point cloud file
PromptStringOptions pso =
new PromptStringOptions("\nEnter name for point cloud: ");
pso.AllowSpaces = true;
PromptResult pr = ed.GetString(pso);
if (pr.Status != PromptStatus.OK)
return;
string name = pr.StringResult;
// Find out whether to run this in the foreground or not
PromptKeywordOptions pko =
new PromptKeywordOptions(
"\nRun in the background?"
);
pko.AllowNone = true;
pko.Keywords.Add("Yes");
pko.Keywords.Add("No");
pko.Keywords.Default = "Yes";
PromptResult pkr =
ed.GetKeywords(pko);
if (pkr.Status != PromptStatus.OK)
return;
bool background = (pkr.StringResult == "Yes");
// We'll store most local files in the temp folder.
// We get a temp filename, delete the file and
// use the name for our folder
string localPath = Path.GetTempFileName();
File.Delete(localPath);
Directory.CreateDirectory(localPath);
localPath += "\\";
// Decide where to store the resultant scene
string rziPath =
localPath + MakeValidFileName(name) + ".rzi";
// We need the current assmbly's location to find and
// run a few of the utilities we need
string exePath =
Path.GetDirectoryName(
Assembly.GetExecutingAssembly().Location
) + "\\";
string pfcPath = exePath + "PhotoflyClient\\";
if (!File.Exists(pfcPath + "photoflyClient.exe"))
{
ed.WriteMessage(
"\nCould not find the Photofly Client tool: " +
"please make sure it is in the same folder " +
"as the application DLL."
);
return;
}
if (background)
{
// Watch for the output file
if (_fsw != null)
_fsw.Dispose();
_fsw = new FileSystemWatcher(localPath);
_fsw.Created += new FileSystemEventHandler(fsw_Created);
_fsw.EnableRaisingEvents = true;
// Set up our status-bar tray item
if (_ti != null)
_ti.Dispose();
_ti = new TrayItem();
_ti.ToolTipText = "Processing images using Photofly...";
if (_procIcon == null)
{
_procIcon =
new System.Drawing.Icon(exePath + "CloudProcessing.ico");
_readyIcon =
new System.Drawing.Icon(exePath + "CloudReady.ico");
}
// Add our tray item to the status bar
_ti.Icon = _procIcon;
StatusBar sb = Application.StatusBar;
sb.TrayItems.Add(_ti);
// Make sure the status bar refreshes
System.Drawing.Rectangle rect =
new System.Drawing.Rectangle(
sb.Window.Location,
sb.Window.Size
);
InvalidateRect(IntPtr.Zero, ref rect, true);
Application.StatusBar.Update();
ed.WriteMessage(
"\nUploading images from {0} for processing in the " +
"Photofly web service in the background.",
imgPath
);
// Launch the process
ProcessStartInfo psi =
new ProcessStartInfo(
pfcPath + "photoflyClient",
"--dir=\"" + imgPath + "\" --output=\"" + rziPath + "\""
);
psi.CreateNoWindow = false;
psi.WindowStyle = ProcessWindowStyle.Hidden;
Process.Start(psi);
}
else
{
// Launch the process, redirecting the output to our
// command-line
Process pfc = new Process();
pfc.StartInfo.FileName = pfcPath + "photoflyClient.exe";
pfc.StartInfo.Arguments =
"--dir=\"" + imgPath + "\" --output=\"" + rziPath + "\"";
pfc.StartInfo.UseShellExecute = false;
pfc.Start();
pfc.WaitForExit();
doc.SendStringToExecute(
"_.IP " + rziPath + "\n",
false,
false,
false
);
}
}
// Our results file has been created
void fsw_Created(object sender, FileSystemEventArgs e)
{
// Disable future notifications
FileSystemWatcher fsw = sender as FileSystemWatcher;
if (fsw != null)
fsw.EnableRaisingEvents = false;
// Show a status bar balloon allowing insertion of our file
if (e.FullPath.EndsWith(".rzi"))
StatusBarBalloon(e.FullPath);
}
public void StatusBarBalloon(string filename)
{
const string title =
"Photofly Upload Complete";
const string msg =
"The point cloud file is ready to be indexed and attached.";
TrayItemBubbleWindow bw = new TrayItemBubbleWindow();
bw.Title = title;
bw.HyperText = filename;
bw.Text = msg;
bw.IconType = IconType.Information;
_ti.Icon = _readyIcon;
_ti.ToolTipText = "Photofly Client";
_ti.ShowBubbleWindow(bw);
Application.StatusBar.Update();
bw.Closed +=
delegate(
object o,
TrayItemBubbleWindowClosedEventArgs args
)
{
// Use a try-catch block, as an exception
// will occur when AutoCAD is closed with
// one of our bubbles open
try
{
Application.StatusBar.TrayItems.Remove(_ti);
Application.StatusBar.Update();
System.Windows.Forms.Application.DoEvents();
}
catch
{ }
if (args.CloseReason ==
TrayItemBubbleWindowCloseReason.HyperlinkClicked)
{
SendCommandToAutoCAD("_.IP " + filename + "\n");
}
};
}
[CommandMethod("IP")]
public void ImportFromPhotofly()
{
Document doc =
Application.DocumentManager.MdiActiveDocument;
Editor ed = doc.Editor;
PromptStringOptions pso =
new PromptStringOptions("\nEnter RZI location: ");
pso.AllowSpaces = true;
PromptResult pr = ed.GetString(pso);
if (pr.Status != PromptStatus.OK)
return;
string rziPath = pr.StringResult;
string name = Path.GetFileNameWithoutExtension(rziPath);
// Paths for our temporary files
string localPath = Path.GetDirectoryName(rziPath) + "\\";
string txtPath = localPath + "points.txt";
string lasPath = localPath + "points.las";
// Our PCG file will be stored under My Documents
string outputPath =
Environment.GetFolderPath(
Environment.SpecialFolder.MyDocuments
) + "\\Photofly Point Clouds\\";
if (!Directory.Exists(outputPath))
Directory.CreateDirectory(outputPath);
// We'll use the title as a base filename for the PCG,
// but will use an incremented integer to get an unused
// filename
int cnt = 0;
string pcgPath;
do
{
pcgPath =
outputPath + name +
(cnt == 0 ? "" : cnt.ToString()) + ".pcg";
cnt++;
}
while (File.Exists(pcgPath));
// The path to the txt2las tool will be the same as the
// executing assembly (our DLL)
string exePath =
Path.GetDirectoryName(
Assembly.GetExecutingAssembly().Location
) + "\\";
if (!File.Exists(exePath + "txt2las.exe"))
{
ed.WriteMessage(
"\nCould not find the txt2las tool: please make sure it " +
"is in the same folder as the application DLL."
);
return;
}
ed.WriteMessage(
"\nCreating a LAS from the downloaded points.\n"
);
if (File.Exists(rziPath))
{
string xslPath = exePath + "rzi2txt.xslt";
XslCompiledTransform xct = new XslCompiledTransform();
xct.Load(xslPath);
xct.Transform(rziPath, txtPath);
}
else
{
ed.WriteMessage("\nScene not generated.");
}
if (File.Exists(txtPath))
{
File.Delete(rziPath);
// Use the txt2las utility to create a .LAS
// file from our text file
ProcessStartInfo psi =
new ProcessStartInfo(
exePath + "txt2las",
"-i \"" + txtPath +
"\" -o \"" + lasPath +
"\" -parse xyz"
);
psi.CreateNoWindow = false;
psi.WindowStyle = ProcessWindowStyle.Hidden;
// Wait for the process to exit
try
{
using (Process p = Process.Start(psi))
{
p.WaitForExit();
}
}
catch
{ }
// If there's a problem, we return
if (!File.Exists(lasPath))
{
ed.WriteMessage(
"\nError creating LAS file."
);
return;
}
File.Delete(txtPath);
ed.WriteMessage(
"Indexing the LAS and attaching the PCG.\n"
);
// Index the .LAS file, creating a .PCG
string lasLisp = lasPath.Replace('\\', '/'),
pcgLisp = pcgPath.Replace('\\', '/');
doc.SendStringToExecute(
"(command \"_.POINTCLOUDINDEX\" \"" +
lasLisp + "\" \"" +
pcgLisp + "\")(princ) ",
false, false, false
);
// Attach the .PCG file
doc.SendStringToExecute(
"_.WAITFORFILE2 \"" +
pcgLisp + "\" \"" +
lasLisp + "\" " +
"(command \"_.-POINTCLOUDATTACH\" \"" +
pcgLisp +
"\" \"0,0\" \"1\" \"0\")(princ) ",
false, false, false
);
doc.SendStringToExecute(
"_.ZOOM _E ",
false, false, false
);
}
}
// Return whether a file is accessible
private bool IsFileAccessible(string filename)
{
// If the file can be opened for exclusive access it means
// the file is accesible
try
{
FileStream fs =
File.Open(
filename, FileMode.Open,
FileAccess.Read, FileShare.None
);
using (fs)
{
return true;
}
}
catch (IOException)
{
return false;
}
}
// A command which waits for a particular PCG file to exist
[CommandMethod("WAITFORFILE2", CommandFlags.NoHistory)]
public void WaitForFileToExist()
{
Document doc =
Application.DocumentManager.MdiActiveDocument;
Editor ed = doc.Editor;
PromptResult pr = ed.GetString("Enter path to PCG: ");
if (pr.Status != PromptStatus.OK)
return;
string pcgPath = pr.StringResult.Replace('/', '\\');
pr = ed.GetString("Enter path to LAS: ");
if (pr.Status != PromptStatus.OK)
return;
string lasPath = pr.StringResult.Replace('/', '\\');
ed.WriteMessage(
"\nWaiting for PCG creation to complete...\n"
);
// Check the write time for the PCG file...
// if it hasn't been written to for at least half a second,
// then we try to use a file lock to see whether the file
// is accessible or not
const int ticks = 50;
TimeSpan diff;
// First loop is to see when writing has stopped
// (better than always throwing exceptions)
while (true)
{
if (File.Exists(pcgPath))
{
DateTime dt = File.GetLastWriteTime(pcgPath);
diff = DateTime.Now - dt;
if (diff.Ticks > ticks)
break;
}
System.Windows.Forms.Application.DoEvents();
}
// Second loop will wait until file is finally accessible
// (by calling a function that requests an exclusive lock)
int inacc = 0;
while (true)
{
if (IsFileAccessible(pcgPath))
break;
else
inacc++;
System.Windows.Forms.Application.DoEvents();
}
ed.WriteMessage("\nFile inaccessible {0} times.", inacc);
try
{
CleanupTmpFiles(lasPath);
}
catch
{ }
}
// Just use the Win32 API to communicate with AutoCAD.
// We simply need to send a command string, so this
// approach avoids a dependency on AutoCAD's COM
// interface.
private void SendCommandToAutoCAD(string toSend)
{
const int WM_COPYDATA = 0x4A;
COPYDATASTRUCT cds = new COPYDATASTRUCT();
cds.dwData = new IntPtr(1);
string data = toSend + "\0";
cds.cbData = data.Length * Marshal.SystemDefaultCharSize;
cds.lpData = Marshal.StringToCoTaskMemAuto(data);
SendMessageW(
Application.MainWindow.Handle,
WM_COPYDATA,
IntPtr.Zero,
ref cds
);
Marshal.FreeCoTaskMem(cds.lpData);
}
private void CleanupTmpFiles(string txtPath)
{
if (File.Exists(txtPath))
File.Delete(txtPath);
Directory.Delete(
Path.GetDirectoryName(txtPath)
);
}
private static string MakeValidFileName(string name)
{
string invChars =
Regex.Escape(new string(Path.GetInvalidFileNameChars()));
string invRegEx = string.Format(@"[{0}]", invChars + ".");
return Regex.Replace(name, invRegEx, "-");
}
}
}
Here’s the very simple XSLT stylesheet that is used to generate comma-delimited, plain text from the resultant RZI file:
Here’s what happens when we run the UP command (for “Upload to Photofly”) against the set of images used in the last post.
… and then provide a name for the point cloud (such as “Kean’s head”).
We then get to choose whether to run the client in the foreground (which basically launches the executable visibly and blocks AutoCAD’s UI while waiting for it to complete) or the background. Here’s what we’ll see if we run it in the foreground:
Like other comparable activities – such as PUBLISH or POINTCLOUDINDEX – when the command runs in background mode an icon gets added to the AutoCAD application’s status bar in the bottom right corner of the main frame. I would have liked to have animated the icon, but that’s currently not possible when using a managed language such as C#, at least as far as I can tell. And I didn’t want to drop down into C++ just for this.
It’s from here that a bubble notification gets displayed which can be used to complete the import:
Whichever route we chose to generate the point cloud – background or foreground – the results should be the same:
If you have AutoCAD 2011 and are interested in comparing the results of importing my head using Photofly and Photosynth, you can find the PCG files here: Kean’s head using Photofly, Kean’s head using Photosynth. Simply use POINTCLOUDATTACH to bring them into your AutoCAD session.
I’ll certainly be back with more on this once we have a version of this tool we can publish. In the meantime, in the next post you’ll be able to see the application in action during an all-new ADN DevCast.
Subscribe via RSS