Through the Interface: Importing Photosynth point clouds into AutoCAD 2011 Part 4

Kean Walmsley

May 2015

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30


« "AutoCAD 2011 New APIs" webcast recording | Main | May’s Plugin of the Month: Spiro for AutoCAD »

April 30, 2010

Importing Photosynth point clouds into AutoCAD 2011 – Part 4

In the previous posts in this series we introduced a command that downloaded and imported point clouds from, we introduced a WinForms user interface on top of it and then replaced that UI with one implemented using WPF.

As threatened last time, we’re now going to make some efficiency improvements in the original command implementation.

In our previous implementation we were blindly asking for files, one after the other, and using failure to indicate when we’d reached the end. Which was fine, but it limited us in a few ways: we could not reliably parallelize this otherwise highly parallelizable operation, and we couldn’t report accurate progress back to the user (as we didn’t know when it was all going to end).

Thanks to a note from Nate Lawrence and another look at Christoph Hausner’s Photosynth Point Cloud Exporter project, I was able to work out how to get this information from the Photosynth web service. The beauty of getting this information is that we now know exactly what files we need to download and can fire them off as asynchronous tasks.

The best way I know of managing this kind of activity is by integrating F# into your project, and this is – in my opinion – one of the absolutely compelling benefits of the F# language: it’s just so easy to capture the logic of “Asynchronous Workflows”, such as this, and to leave the F# subsystem to execute them as efficiently as it can. And, as we’ll see, for a task where we’re downloading and processing multiple files there are huge performance benefits versus performing this sequentially.

Before we look at the code, a few notes on connecting to the Photosynth web service. F# projects – at least with the April 2010 CTP I’m using with VS 2008 – do not have IDE support for adding web service references, so I decided to keep this “discovery” activity in C#.

Adding service references to a C# project is easy – we right-click inside the Solution Explorer and select “Add Service Reference…”, copying/pasting the web service URL ( into the Address bar:

Add a reference to the Photosynth web service

After giving it a name, we select OK to see some details:

Our Photosynth web service

Here we see the GetCollectionData() method, which is the one we’re going to use in this application.

The latest source project is available here. I’ve included the two files which have either been introduced (F#) or heavily updated (C#) below, but there have been a few other miscellaneous changes to other files in the project.

Let’s start by looking at the C# code in our updated import-photosynth.cs file:

using Autodesk.AutoCAD.ApplicationServices;

using Autodesk.AutoCAD.DatabaseServices;

using Autodesk.AutoCAD.EditorInput;

using Autodesk.AutoCAD.Runtime;

using Autodesk.AutoCAD.Geometry;

using Autodesk.AutoCAD.Colors;

using System.Windows.Threading;

using System.Threading;

using System.Text.RegularExpressions;

using System.ServiceModel;

using System.Reflection;

using System.Net;

using System.IO;

using System.Diagnostics;

using System;

using DemandLoading;

using ImportPhotosynth.PhotosynthService;

using Newtonsoft.Json;

using Newtonsoft.Json.Linq;


namespace ImportPhotosynth


  public class Appl : IExtensionApplication


    public void Initialize()








      { }



    public void Terminate()






  public class Commands


    const string exeName = "ADNPlugin-BrowsePhotosynth2";


    static Process _p = null;


    static public void Cleanup()


      if (_p != null)


        if (!_p.HasExited)




        _p = null;




    static public void CleanupOnStartup()


      bool first = true;


      foreach (Process proc in Process.GetProcesses())


        if (proc.ProcessName.Contains(exeName))


          if (first)


            if (System.Windows.Forms.MessageBox.Show(

                  "Instances of browser executable found running. " +

                  "Would you like them closed?",

                  "Import Photosynth",


                ) != System.Windows.Forms.DialogResult.Yes)




            first = false;







    [CommandMethod("BP", CommandFlags.Session)]

    public void BrowsePhotosynth()


      Document doc =


      Database db = doc.Database;

      Editor ed = doc.Editor;




      string exePath =



        ) + "\\";


      if (!File.Exists(exePath + exeName + ".exe"))



          "\nCould not find the {0} tool: please make sure " +

          "it is in the same folder as the application DLL.",






      // Launch our browser window with the AutoCAD's handle

      // so that we can receive back command strings


      ProcessStartInfo psi =

        new ProcessStartInfo(

          exePath + exeName,

          " " + Application.MainWindow.Handle


      _p = Process.Start(psi);



    [CommandMethod("IMPORTPHOTOSYNTH", CommandFlags.NoHistory)]

    public void ImportPhotosynth()


      Document doc =


      Database db = doc.Database;

      Editor ed = doc.Editor;

      HostApplicationServices ha =



      PromptResult pr =


          "Enter URL of first Photosynth point cloud: "


      if (pr.Status != PromptStatus.OK)



      string path = pr.StringResult;


      pr =


          "Enter name of Photosynth point cloud: "


      if (pr.Status != PromptStatus.OK)



      string name = pr.StringResult;


      // The root path has "points_0_0.bin" on the end.

      // Strip off the last 5 characters ("0_0.bin"), so

      // that we can compose the sequence of URLs needed

      // for each of the point cloud files (usually

      // going up to about "points_0_23.bin")


      if (path.Length > 5)

        path = path.Substring(0, path.Length - 7);


      // We'll store most local files in the temp folder.

      // We get a temp filename, delete the file and

      // use the name for our folder


      string localPath = Path.GetTempFileName();



      localPath += "\\";


      // Paths for our temporary files


      string txtPath = localPath + "points.txt";

      string lasPath = localPath + "points.las";


      // Our PCG file will be stored under My Documents


      string outputPath =



        ) + "\\Photosynth Point Clouds\\";


      if (!Directory.Exists(outputPath))



      string colId = ExtractCollectionId(path);


      // We'll use the title as a base filename for the PCG,

      // but will use an incremented integer to get an unused

      // filename


      int cnt = 0;

      string pcgPath;



        pcgPath =

          outputPath + MakeValidFileName(name) +

          (cnt == 0 ? "" : cnt.ToString()) + ".pcg";



      while (File.Exists(pcgPath));     


      // The path to the txt2las tool will be the same as the

      // executing assembly (our DLL)


      string exePath =



        ) + "\\";


      if (!File.Exists(exePath + "txt2las.exe"))



          "\nCould not find the txt2las tool: please make sure it " +

          "is in the same folder as the application DLL."





      // We now access the Photosynth web service to get the size of

      // the cloud(s) we want to download and process



        "\nAccessing Photosynth web service to get information on "+

        "\"{0}\" point cloud(s)...\n", name



      // We're interested in two URLs


      string dzcUrl, jsonUrl;


      // Perform manual binding, to avoid having to add binding info

      // into acad.exe.config


      BasicHttpBinding binding = new BasicHttpBinding();

      EndpointAddress address =

        new EndpointAddress(




      // Create our SOAP client


      PhotosynthServiceSoapClient soapClient =

        new PhotosynthServiceSoapClient(binding, address);

      using (soapClient)




          // Get the data associated with our Photosynth cloud(s)


          CollectionResult colRes =

            soapClient.GetCollectionData(new Guid(colId), false);

          dzcUrl = colRes.DzcUrl;

          jsonUrl = colRes.JsonUrl;


        catch (FormatException fex)


          ed.WriteMessage("\nInvalid URL: {0}", fex.Message);



        catch (EndpointNotFoundException ex)



            "\nCould not connect to Photosynth web service: {0}",







      if (jsonUrl == null || dzcUrl == null)



          "\nUnable to find information about this point cloud " +

          "via the Photosynth web service."




      string jsonData;


      // All being well we should now be able to download and process

      // the data about our cloud(s)


      using (WebClient webClient = new WebClient())

        jsonData = webClient.DownloadString(jsonUrl);


      // Extract our point cloud dimension information, as per:



      JObject jObject = JObject.Parse(jsonData);

      JToken cols = jObject["l"] ?? jObject["collections"];

      JToken col = cols[colId] ?? cols[string.Empty];

      JToken numCoordSystems = col["_num_coord_systems"];

      JToken coordSystems = col["x"] ?? col["coord_systems"];


      // Create the array of integers representing these

      // dimensions


      int totalClouds = (int)numCoordSystems;

      int[] dims = new int[totalClouds];


      // Variables to count the number of files and points


      int totalFiles = 0;

      long totalPoints = 0;


      // Populate our dimensions list and count the files


      for (int i = 0; i < totalClouds; i++)


        JToken cs = coordSystems[Convert.ToString(i)];

        JToken pc = cs["k"] ?? cs["pointcloud"];

        if (pc != null)


          string s = (string)pc[0];


          if (!string.IsNullOrEmpty(s))


            JToken binFileCount = pc[1];

            int fileCount = (int)binFileCount;

            dims[i] = fileCount;

            totalFiles += fileCount;





      // Report back what we've found, thus far



        "\n{0} point cloud{1} found across {2} file{3}.\n",

        totalClouds, totalClouds == 1 ? "" : "s",

        totalFiles, totalFiles == 1 ? "" : "s"



      // Start the progress meter for our processing

      // operation


      ProgressMeter pm = new ProgressMeter();

      using (pm)



        pm.Start("Downloading/processing Photosynth points");




          // If the current SynchronizationContext is null

          // (which appears to be the case when not called

          // from the debugger) then create one and set it


          // We will need this to coordinate UI update events

          // back with this thread


          if (SynchronizationContext.Current == null)


            DispatcherSynchronizationContext context =

              new DispatcherSynchronizationContext(








          // Create our processor object


          ProcessPhotosynth.PointCloudProcessor pcp =

            new ProcessPhotosynth.PointCloudProcessor();


          // Capture the start time


          DateTime start = DateTime.Now;


          // When each file is processed, write a message

          // to the command-line and update the progress


          pcp.JobCompleted +=

            delegate(object sender, Tuple<string, int> args)



                "\nProcessed {0} containing {1} points.",

                args.Item1, args.Item2





          // Process our point cloud(s)


          pcp.ProcessPointCloud(path, dims, txtPath);


          // The above function launches a set of asynchronous

          // tasks and returns. We need to loop while

          // processing UI events until the tasks are complete


          while (!pcp.IsComplete)





          // Now we can find out the results


          totalPoints = pcp.TotalPoints;


          // And calculate/report the elapsed time


          TimeSpan elapsed = DateTime.Now - start;



            "\nImported {0} points from {1} file{2} in {3}.\n",

            totalPoints, totalFiles, totalFiles == 1 ? "" : "s",





        catch (System.Exception ex)



            "\nException occurred: {0}", ex.Message




        // Stop the progress meter





      if (totalPoints > 0)


        // Use the txt2las utility to create a .LAS

        // file from our text file


        ProcessStartInfo psi =

          new ProcessStartInfo(

            exePath + "txt2las",

            "-i \"" + txtPath +

            "\" -o \"" + lasPath +

            "\" -parse xyzRGB"


        psi.CreateNoWindow = false;

        psi.WindowStyle = ProcessWindowStyle.Hidden;


        // Wait up to 20 seconds for the process to exit




          using (Process p = Process.Start(psi))






        { }


        // If there's a problem, we return


        if (!File.Exists(lasPath))



            "\nError creating LAS file."







          "Indexing the LAS and attaching the PCG.\n"



        // Index the .LAS file, creating a .PCG


        string lasLisp = lasPath.Replace('\\', '/'),

              pcgLisp = pcgPath.Replace('\\', '/');



          "(command \"_.POINTCLOUDINDEX\" \"" +

          lasLisp + "\" \"" +

          pcgLisp + "\")(princ) ",

          false, false, false



        // Attach the .PCG file



          "_.WAITFORFILE \"" +

          pcgLisp + "\" \"" +

          lasLisp + "\" " +

          "(command \"_.-POINTCLOUDATTACH\" \"" +

          pcgLisp +

          "\" \"0,0\" \"1\" \"0\")(princ) ",

          false, false, false




          "_.-VISUALSTYLES _C _Conceptual _.ZOOM _E ",

          false, false, false





    private string ExtractCollectionId(string path)


      const string synthTag = ".synth_files";


      string colId = "";


      if (path.Contains(synthTag))


        string start =

          path.Substring(0, path.IndexOf(synthTag));

        if (start.Length > 36)

          colId = start.Substring(start.Length - 36);


      return colId;



    // A command which waits for a particular PCG file to exist


    [CommandMethod("WAITFORFILE", CommandFlags.NoHistory)]

    public void WaitForFileToExist()


      Document doc =


      Database db = doc.Database;

      Editor ed = doc.Editor;

      HostApplicationServices ha =



      PromptResult pr = ed.GetString("Enter path to PCG: ");

      if (pr.Status != PromptStatus.OK)


      string pcgPath = pr.StringResult.Replace('/', '\\');


      pr = ed.GetString("Enter path to LAS: ");

      if (pr.Status != PromptStatus.OK)


      string lasPath = pr.StringResult.Replace('/', '\\');


      // Check the write time for the PCG file...

      // if it hasn't been written to for at least four seconds,

      // we can continue


      const int numSecs = 4;

      TimeSpan span = new TimeSpan(0,0,numSecs);


      TimeSpan diff;

      while (true)


        if (File.Exists(pcgPath))


          DateTime dt = File.GetLastWriteTime(pcgPath);

          diff = DateTime.Now - dt;

          if (diff.Ticks > span.Ticks)











      { }



    private void CleanupTmpFiles(string txtPath)


      if (File.Exists(txtPath))







    private static string MakeValidFileName(string name)


      string invChars =

        Regex.Escape(new string(Path.GetInvalidFileNameChars()));

      string invRegEx = string.Format(@"[{0}]", invChars + ".");

      return Regex.Replace(name, invRegEx, "-");




A few comments on the code:

  • When running this code a lot in the debugger, I found we were getting lots of hidden instances of our browser executable (as we’re killing it rather than allowing it to exit naturally, due to a bug in the csExWb2 component), so I decided to provide some automatic clean-up functionality on startup of the application
  • We’ve removed a lot of code dealing with the downloading/processing of point data – this is all now taken care of by the F#-implemented PointCloudProcessor object
  • This C# code now depends on an additional library called Json.NET (as did the exporter from which I borrowed the code) to parse JSON-formatted information we download from the Photosynth web service
  • We have a situation where we want to perform asynchronous operations on arbitrary threads, but these operations need to report back to the UI thread in order for us to write text to the command-line and to update AutoCAD’s progress meter. For this we have to make sure we have a valid SynchronizationContext set-up, which will be used from our F# code

Here’s the F# code which takes care of defining and running these asynchronous operations:

module ProcessPhotosynth


open System

open System.IO

open System.Net

open System.Text

open System.Threading


// We need the SynchronizationContext of the UI thread,

// to allow us to make sure our UI update events get

// processed correctly in the calling application


let mutable syncContext : SynchronizationContext = null


// Asynchronous Worker courtesy of Don Syme:


//  parallel-design-patterns-in-f-reporting-progress-with-

//  events-plus-twitter-sample.aspx


type Agent<'T> = MailboxProcessor<'T>


type SynchronizationContext with


  // A standard helper extension method to raise an event on

  // the GUI thread


  member syncContext.RaiseEvent (event: Event<_>) args =

    syncContext.Post((fun _ -> event.Trigger args),state=null)


type AsyncWorker<'T>(jobs: seq<Async<'T>>) =


  // Each of these lines declares an F# event that we can raise


  let allCompleted  = new Event<'T[]>()

  let error        = new Event<System.Exception>()

  let canceled      = new Event<System.OperationCanceledException>()

  let jobCompleted  = new Event<int * 'T>()


  let cancellationCapability = new CancellationTokenSource()


  // Start an instance of the work


  member x.Start() =                                                     


    // Capture the synchronization context to allow us to raise

    // events back on the GUI thread


    if syncContext = null then

      syncContext <- SynchronizationContext.Current


    if syncContext = null then



          "Synchronization context is null."))     


    // Mark up the jobs with numbers


    let jobs = jobs |> Seq.mapi (fun i job -> (job,i+1))


    let work =


      [ for (job,jobNumber) in jobs ->

          async { let! result = job


                    jobCompleted (jobNumber,result)

                  return result } ]




      (fun res -> syncContext.RaiseEvent allCompleted res),

      (fun exn -> syncContext.RaiseEvent error exn),

      (fun exn -> syncContext.RaiseEvent canceled exn ),



  member x.CancelAsync() =



  // Raised when a particular job completes


  member x.JobCompleted = jobCompleted.Publish


  // Raised when all jobs complete


  member x.AllCompleted = allCompleted.Publish


  // Raised when the composition is cancelled successfully


  member x.Canceled = canceled.Publish


  // Raised when the composition exhibits an error


  member x.Error = error.Publish


type PointCloudProcessor() =


  // Mutable state to track progress and results


  let mutable jobsComplete = 0

  let mutable totalJobs = 0

  let mutable totalPoints = 0


  // Event to allow caller to update the UI


  let jobCompleted  = new Event<string * int>()


  // Function to access a stream asynchronously


  let httpAsync(url:string) =


    async {

      let req = WebRequest.Create(url)

      let! rsp = req.AsyncGetResponse()

      return rsp.GetResponseStream()



  // Functions to read data from our point stream


  let rec readCompressedInt (i:int) (br:BinaryReader) =

    let b = br.ReadByte()

    let i = (i <<< 7) ||| ((int)b &&& 127)

    if (int)b < 128 then

      readCompressedInt i br




  let readBigEndianFloat (br:BinaryReader) =

    let b = br.ReadBytes(4)

    BitConverter.ToSingle( [| b.[3]; b.[2]; b.[1]; b.[0] |], 0)


  let readBigEndianShort (br:BinaryReader) =

    let b1 = br.ReadByte()

    let b2 = br.ReadByte()

    ((uint16)b2 ||| ((uint16)b1 <<< 8))


  // Recursive function to read n points from our stream

  // (We use an accumulator variable to enable tail-call

  // optimization)


  let rec readPoints acc n br =


    if n <= 0 then




      // Read our coordinates


      let x = readBigEndianFloat br

      let y = readBigEndianFloat br

      let z = readBigEndianFloat br


      // Read and extract our RGB values


      let rgb = readBigEndianShort br


      let r = (rgb >>> 11) * 255us / 31us

      let g = ((rgb >>> 5) &&& 63us) * 255us / 63us

      let b = (rgb &&& 31us) * 255us / 31us


      readPoints ((x,y,z,r,g,b) :: acc) (n-1) br


  // Function to extract the various point information

  // from a stream cooresponding to a single poinr file


  let extractPoints br =


    // First information is the file version

    // (for now we support version 1.0 only)


    let majVer = readBigEndianShort br

    let minVer = readBigEndianShort br


    if (int)majVer <> 1 || (int)minVer <> 0 then




      // Clear some header bytes we don't care about


      let n = readCompressedInt 0 br

      for i in 0..(int)n-1 do

        let m = readCompressedInt 0 br

        for j in 0..(int)m-1 do

          readCompressedInt 0 br |> ignore

          readCompressedInt 0 br |> ignore


      // Find out the number of points in the file


      let npts = readCompressedInt 0 br


      // Read and return the points


      readPoints [] npts br


  // Recursive function to create a string from a list

  // of points. Our accumulator is a StringBuilder,

  // which is the most efficient way to collate a

  // string


  let rec pointsToString (acc : StringBuilder) pts =

    match pts with

    | [] -> acc.ToString()

    | (x,y,z,r,g,b) :: t ->       

      acc.AppendFormat("{0},{1},{2},{3},{4},{5}\n", x, y, z, r, g, b)

        |> ignore

      pointsToString acc t


  // Recursive function to write a list of points to file


  let rec addPointsToFile (sw : StreamWriter) pts =

    match pts with

    | [] -> ()

    | (x,y,z,r,g,b) :: t ->

      sw.WriteLine("{0},{1},{2},{3},{4},{5}", x, y, z, r, g, b)

      addPointsToFile sw t


  // Expose an event that's subscribable from C#/VB



  member x.JobCompleted = jobCompleted.Publish


  // Property to indicate that we're done


  member x.IsComplete = (jobsComplete = totalJobs)


  // Property to return the results


  member x.TotalPoints = totalPoints


  // Our main function to download and process the point

  // cloud(s) associated with a particular Photosynth


  member x.ProcessPointCloud baseUrl dims txtPath =


    // A local function to add the URL prefix to each file


    let getLocalFilename file = baseUrl + file


    // Generate our list of files from the list of dimensions

    // of the various point clouds


    // Each entry in dims corresponds to the number of files:

    //  dims[0] = 5 means "points_0_0.bin" .. "points_0_4.bin"

    //  dims[6] = 3 means "points_6_0.bin" .. "points_6_2.bin"


    let files =


        (fun i d ->

 (fun j -> sprintf "%d_%d.bin" i j) [| 0..d-1 |]



        |> Array.concat

        |> List.ofArray


    // Set/reset mutable state


    totalJobs <- files.Length

    jobsComplete <- 0


    // Open the local, temporary text file to hold our points


    let t = new FileInfo(txtPath)

    let sw = t.Create()


    // An agent to store our points in the file...

    // Loops and receives messages, so that we ensure we don't

    // have a conflict of simultaneous writes


    let fileAgent =

      Agent.Start(fun inbox ->

        async { while true do

                  let! (msg : string) = inbox.Receive()

                  do! sw.AsyncWrite(Encoding.ASCII.GetBytes(msg)) })


    // Our basic asynchronous task to process a file, returning

    // the number of points


    let processFile (file:string) =

      async {

        let! stream = httpAsync file

        use reader = new BinaryReader(stream)

        let pts = extractPoints reader

        pointsToString (new StringBuilder()) pts |> fileAgent.Post

        return file, pts.Length



    // Our jobs are a set of tasks, one for each file


    let jobs =

      [for file in files ->

        getLocalFilename file

          |> processFile



    // Create our AsyncWorker for our jobs


    let worker = new AsyncWorker<_>(jobs)


    // Raise an event when each file is processed and update

    // our internal state


    worker.JobCompleted.Add(fun (jobNumber, (url , ptnum)) ->

      let file = url.Substring(url.LastIndexOf('/')+1)

      jobsComplete <- jobsComplete + 1

      syncContext.RaiseEvent jobCompleted (file, ptnum)


      // If the last job, close our temporary file


      if x.IsComplete then





    // Once we're all done, set the results as state to be

    // accessed by our calling routine


    worker.AllCompleted.Add(fun results ->

        totalPoints <- Array.sumBy snd results )


    // Now start the work



Some comments on this code:

  • A big chunk of this implementation has been copied verbatim from Don Syme’s excellent post on reporting progress from asynchronous tasks. The main change to Don’s AsyncWorker implementation is our need to fail should we be unable to get a valid SynchronizationContext for our main thread: if this doesn’t work then we need to fail (gracefully), as we will not be able to report our progress via a new context executing in the .NET thread pool
  • Otherwise we have a number of tail-recursive functions (a topic I’ve discussed previously), to avoid both iterative code and stack overflows
  • We also expose an event which is subscribed to in our C# calling code. This means we don’t need any dependency on AutoCAD libraries in the F# project: to update the command-line and progress meter we simply need to fire that event from the UI thread and let the code execute from the project with the appropriate assembly references
  • We’re using another asynchronous concept to manage our writing to the local text file for our point data: we’re using an agent to manage this

That’s really about it in terms of the changes. Let’s take it all for a spin.

We’ll start by running our code against the largest, most detailed Photosynth I’ve seen, another suggestion from Nate Lawrence: Mark Willis’ Tres Yonis synth. This is an impressively detailed synth: its point cloud contains 1.1M points and it was defined by 534 high-resolution photos.

The epic Another Tres Yonis Synth

Downloading and processing these files sequentially takes around 5 minutes (I just measure it at 4:48, but have also see it taking around five and a half).

When we perform the same operation using our new, improved application, it now takes a hair over 35 seconds! That’s less than 1/8th of the time. Here’s this magnificent Photosynth’s point cloud inside AutoCAD:

The epic Another Tres Yonis point cloud inside AutoCAD 2011

Well, that’s it for today. I have a few other places I want to invest time working with (or should that be “playing with”? :-) this technology. I want to go through the process of using Photosynth to capture a real-world model and then work on it inside AutoCAD, modelling the captured geometry. I also want to compare the results of this approach with that of working with point clouds generated by a 3D laser scanner (our friends at FARO are hopefully providing one sometime in the next few weeks, which I’m very excited about). This is a really exciting area, and you can expect me to spend more time on it over the coming months (although I’ll continue to address other areas, too, for those that find this stuff boring :-).

blog comments powered by Disqus


10 Random Posts