October 2014

Sun Mon Tue Wed Thu Fri Sat
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31  










« Integrating Leap Motion and AutoCAD: Command Interaction | Main | Integrating Leap Motion and AutoCAD: Review »

January 16, 2013

Integrating Leap Motion and AutoCAD: 3D Geometry

After cranking out a post per day so far this week on my exploratory integration of AutoCAD with the Leap Motion controller, it’s time to wrap up the technical portion of my “Leap Week” (a bit like a Leap Year, geddit? ;-) with a nice, juicy topic: creating 3D geometry inside AutoCAD using the Leap Motion controller.

The Leap Motion controller has a couple of key selling points for interacting in 3D space. It’s both highly reactive – they’ve done a great job of minimising any processing lag to allow you to build highly responsive systems – and very accurate.

The accuracy bit makes me think back to when we last had “accurate” input from a peripheral device, when people used to digitise 2D using a digitising tablet. This process gradually died out after the introduction of the mouse: a less precise way of working, but perfectly acceptable when used in conjunction with drawing aids such as grid and object snapping.

The Leap Motion controller is indeed very accurate, but – aside from its application for navigation, etc. – I think it’ll primarily be of interest to people doing rough, conceptual 3D sketching and sculpting rather than modeling precisely in 3D. The accuracy is there, but it’s just really hard to keep your hand steady enough to make use of it. You can do things to mitigate this (beyond not drinking too much the night before ;-), such as having your code only accept points that are a certain distance from the last one. While this can help cut out a fair amount of “jitter”, I don’t personally feel that the device’s accuracy is going to be a compelling selling point for people who really care about precision.

On to the implementation, then…

I had a bit of a head-start on coding this one, as I’d used a similar approach when creating 3D geometry based on input from Kinect. I created a new Jig for this, integrating the navigation code from Tuesday’s post into it.

A cursor – a sphere – will appear at the tip of the primary “pointable” (which could be a tool or a finger) when either just one or two get detected. There’s a little bit of magic involved in transforming the coordinates coming from the Leap Motion controller to make sure they more-or-less match up with the current view: it’s a little strange to be modeling in 3D when the coordinates are inverted, for instance.

The user can then use the Jig’s keyword input to launch the creation of a spline or 3D polyline based on the movements of this cursor, terminating with the Enter key. Navigation can occur at any point during the command, of course.

Here’s the C# code, which essentially replaces that posted on Tuesday (and can still be used in conjunction with the code from Wednesday).

using System;

using System.Threading;

using System.Collections.Generic;

using Autodesk.AutoCAD.ApplicationServices;

using Autodesk.AutoCAD.ApplicationServices.Core;

using Autodesk.AutoCAD.Colors;

using Autodesk.AutoCAD.DatabaseServices;

using Autodesk.AutoCAD.EditorInput;

using Autodesk.AutoCAD.Geometry;

using Autodesk.AutoCAD.GraphicsInterface;

using Autodesk.AutoCAD.Runtime;

using Leap;

 

namespace LeapMotionIntegration

{

  class JigUtils

  {

    // Custom ArcTangent method, as the Math.Atan

    // doesn't handle specific cases

 

    public static double Atan(double y, double x)

    {

      if (x > 0)

        return Math.Atan(y / x);

      else if (x < 0)

        return Math.Atan(y / x) - Math.PI;

      else  // x == 0

      {

        if (y > 0)

          return Math.PI;

        else if (y < 0)

          return -Math.PI;

        else // if (y == 0) theta is undefined

          return 0.0;

      }

    }

 

    // Computes Angle between current direction

    // (vector from last vertex to current vertex)

    // and the last pline segment

 

    public static double ComputeAngle(

      Vector3d dir, Vector3d xdir, Matrix3d ucs

    )

    {

      var v = dir / 2;

      var cos = v.DotProduct(xdir);

      var sin =

        v.DotProduct(

          Vector3d.ZAxis.TransformBy(ucs).CrossProduct(xdir)

        );

      return Atan(sin, cos);

    }

  }

 

  public class LeapJig : DrawJig

  {

    // Internal state

 

    private Document _doc;

    private Point3dCollection _verts;

    private Point3d _pos;

    private bool _enterPressed;

    private bool _showCursor;

    private bool _drawing;

    private bool _splineVsPline;

 

    public LeapJig(Document doc)

    {

      // Initialise the various members

 

      _doc = doc;

      _verts = new Point3dCollection();

      _pos = Point3d.Origin;

      _enterPressed = false;

      _showCursor = false;

      _drawing = false;

      _splineVsPline = false;

    }

 

    public Point3dCollection Vertices

    {

      get { return _verts; }

      set { _verts = value; }

    }

 

    public Point3d Position

    {

      get { return _pos; }

      set { _pos = value; }

    }

 

    public bool EnterPressed

    {

      get { return _enterPressed; }

      set { _enterPressed = value; }

    }

 

    public bool ShowCursor

    {

      get { return _showCursor; }

      set { _showCursor = value; }

    }

 

    public bool Drawing

    {

      get { return _drawing; }

      set { _drawing = value; }

    }

 

    public bool SplineVsPline

    {

      get { return _splineVsPline; }

      set { _splineVsPline = value; }

    }

 

    protected override SamplerStatus Sampler(JigPrompts prompts)

    {

      // We don't really need a point, but we do need some

      // user input event to allow us to loop, processing

      // for the Leap Motion input

 

      var opts =

        new JigPromptPointOptions(

          "\nEnter to commit polyline/reset view, esc to quit " +

          "or [Pline/Spline]",

          "Pline Spline"

        );

      opts.UserInputControls =

        UserInputControls.NullResponseAccepted;

 

      var pr = prompts.AcquirePoint(opts);

 

      switch (pr.Status)

      {

        case PromptStatus.Keyword:

          {

            _splineVsPline = (pr.StringResult == "Spline");

            _drawing = true;

            break;

          }

        case PromptStatus.None:

          {

            _enterPressed = true;

            return SamplerStatus.OK;

          }

        case PromptStatus.OK:

          {

            ForceMessage();

            return SamplerStatus.OK;

          }

      }

      return SamplerStatus.Cancel;

    }

 

    protected override bool WorldDraw(WorldDraw wd)

    {

      try

      {

        var vtr = _doc.Editor.GetCurrentView();

 

        var hgt = vtr.Height;

        var wid = vtr.Width;

        var min = hgt < wid ? hgt : wid;

        var fac = min / 300;

        var pt = _pos * fac;

        var curRad = hgt / 70;

 

        // Get the displacement to the specified position

 

        var mat = Matrix3d.Displacement(pt.GetAsVector());

 

        // Apply the view's rotation, so we compensate

 

        var rot =

          JigUtils.ComputeAngle(

            vtr.ViewDirection,

            Vector3d.YAxis,

            _doc.Editor.CurrentUserCoordinateSystem

          );

        var rotMat =

          Matrix3d.Rotation(

            rot + Math.PI, Vector3d.ZAxis, Point3d.Origin

          );

        mat = mat.PreMultiplyBy(rotMat);

 

        // Create and draw our solid

 

        if (_showCursor)

        {

          using (var cursor = new Solid3d())

          {

            cursor.CreateSphere(curRad);

            cursor.ColorIndex = 3;

            cursor.TransformBy(mat);

            cursor.WorldDraw(wd);

          }

        }

 

        // If drawing, add vertices when at a reasonable distance

        // from the previous one

 

        if (_drawing)

        {

          if (

            _verts.Count == 0 ||

            _verts[_verts.Count - 1].DistanceTo(pt) > curRad * 5

          )

          {

            _verts.Add(pt.TransformBy(rotMat));

          }

        }

        else if (_verts.Count > 0)

        {

          // If not drawing and there are vertices, clear them

 

          _verts.Clear();

        }

 

        // If we still have vertices in the list...

 

        if (_verts.Count > 1)

        {

          // Create a polyline or spline and draw it

 

          using (var ent = GeneratePathEntity())

          {

            ent.ColorIndex = 2;

            ent.WorldDraw(wd);

          }

        }

      }

      catch { }

 

      return true;

    }

 

    public Entity GeneratePathEntity()

    {

      return

        (_splineVsPline ?

          (Entity)new Spline(_verts, 0, 0.0) :

          new Polyline3d(Poly3dType.SimplePoly, _verts, false)

        );

    }

 

    private void ForceMessage()

    {

      // Set the cursor without ectually moving it - enough to

      // generate a Windows message

 

      var pt = System.Windows.Forms.Cursor.Position;

      System.Windows.Forms.Cursor.Position =

        new System.Drawing.Point(pt.X, pt.Y);

    }

  }

 

  public class Camera

  {

    // Members

 

    private Document _doc = null;

    private ViewTableRecord _vtr = null;

    private ViewTableRecord _initial = null;

 

    public Camera(Document doc)

    {

      _doc = doc;

      _initial = doc.Editor.GetCurrentView();

      _vtr = (ViewTableRecord)_initial.Clone();

    }

 

    // Reset to the initial view

 

    public void Reset()

    {

      _doc.Editor.SetCurrentView(_initial);

      if (_vtr != null)

      {

        _vtr.Dispose();

        _vtr = (ViewTableRecord)_initial.Clone();

      }

      _doc.Editor.Regen();

    }

 

    // Zoom in or out

 

    public void Zoom(double factor)

    {

      // Adjust the ViewTableRecord

 

      _vtr.Height *= factor;

      _vtr.Width *= factor;

 

      // Set it as the current view

 

      _doc.Editor.SetCurrentView(_vtr);

 

      // Zoom requires a regen for the gizmos to update

 

      _doc.Editor.Regen();     

    }

 

    // Pan in the specified direction

 

    public void Pan(double leftRight, double upDown)

    {

      // Adjust the ViewTableRecord

 

      _vtr.CenterPoint =

        _vtr.CenterPoint +

        new Vector2d(

          leftRight * _vtr.Width,

          upDown * _vtr.Height

        );

 

      // Set it as the current view

 

      _doc.Editor.SetCurrentView(_vtr);

    }

 

    // Orbit by angle around axis

 

    public void Orbit(Vector3d axis, double angle)

    {

      // Adjust the ViewTableRecord

 

      _vtr.ViewDirection =

        _vtr.ViewDirection.TransformBy(

          Matrix3d.Rotation(angle, axis, Point3d.Origin)

        );

 

      // Set it as the current view

 

      _doc.Editor.SetCurrentView(_vtr);

    }

 

    public void OrbitVertical(double angle)

    {

      // Adjust the ViewTableRecord

 

      _vtr.ViewDirection =

        _vtr.ViewDirection.TransformBy(

          Matrix3d.Rotation(

            angle,

            _vtr.ViewDirection.GetPerpendicularVector(),

            Point3d.Origin

          )

        );

 

      // Set it as the current view

 

      _doc.Editor.SetCurrentView(_vtr);

    }

  }

 

  public class NavigationListener : Listener

  {

    private Editor _ed;

    private Camera _cam;

    private SynchronizationContext _ctxt;

    private LeapJig _jig;

 

    public NavigationListener(

      Editor ed, Camera cam, SynchronizationContext ctxt, LeapJig jl

    )

    {

      _ed = ed;

      _cam = cam;

      _ctxt = ctxt;

      _jig = jl;

    }

 

    private void WriteLine(String line)

    {

      if (_ed != null)

      {

        _ctxt.Post(a => { _ed.WriteMessage(line + "\n"); }, null);

      }

    }

 

    private void CallOnCam(SendOrPostCallback cb)

    {

      if (_cam == null)

        WriteLine("Cam is null.");

      else

        _ctxt.Post(cb, null);

    }

 

    private void Reset()

    {

      CallOnCam(a => { _cam.Reset(); } );

    }

 

    private void Zoom(double factor)

    {

      CallOnCam(a => { _cam.Zoom(factor); });

    }

 

    private void Pan(double leftRight, double upDown)

    {

      CallOnCam(a => { _cam.Pan(leftRight, upDown); });

    }

 

    private void Orbit(Vector3d axis, double angle)

    {

      CallOnCam(a => { _cam.Orbit(axis, angle); });

    }

 

    private void OrbitVertical(double angle)

    {

      CallOnCam(a => { _cam.OrbitVertical(angle); });

    }

 

    public override void OnInit(Controller controller)

    {

      //WriteLine("\nInitialized");

    }

 

    public override void OnConnect(Controller controller)

    {

      //WriteLine("Connected");

    }

 

    public override void OnDisconnect(Controller controller)

    {

      WriteLine("Disconnected");

    }

 

    public override void OnFrame(Controller controller)

    {

      // Get the most recent frame

 

      var frame = controller.Frame();

      var hands = frame.Hands;

      var numHands = hands.Count;

 

      // Only proceed if we have at least one hand

 

      if (numHands >= 1)

      {

        // Get the first hand and its velocity to check for

        // zoom or pan

 

        var hand = hands[0];

        var handVel = hand.PalmVelocity;

        if (handVel == null)

          handVel = new Vector(0, 0, 0);

 

        // Check if the hand has any fingers

 

        var fingers = hand.Fingers;

        var ptrs = frame.Pointables;

 

        // Only proceed if we see at least two fingers detected

 

        if (fingers.Count == 0 && ptrs.Count == 0)

        {

          _jig.ShowCursor = false;

        }

        else if (fingers.Count > 1)

        {

          _jig.ShowCursor = false;

 

          // Set a flag to see whether we detect zoom or pan

 

          bool zoomOrPan = false;

 

          // Create an AutoCAD vector from the hand's velocity

 

          var handVec =

            new Vector3d(handVel.x, handVel.y, handVel.z);

 

          // Get the largest element and its [absolute] value

 

          var largestDim = handVec.LargestElement;

          var largestVal = handVec[largestDim];

          var largestAbs = Math.Abs(largestVal);

 

          // Depending on the largest value we know to zoom or pan

 

          switch (largestDim)

          {

            case 1:

              if (largestAbs > 250)

              {

                if (frame.Id % 2 == 0)

                {

                  WriteLine(

                    "Zoom " + (largestVal < 0 ? "in" : "out")

                  );

                  Zoom(largestVal > 0 ? 1.1 : 0.9);

                }

                zoomOrPan = true;

              }

              break;

            default:

              if (largestAbs > 100)

              {

                var x = 0.02 * -handVel.x / largestAbs;

                var y = 0.02 * handVel.z / largestAbs;

                WriteLine(

                  "Pan " +

                  (largestDim == 0 ?

                    (largestVal < 0 ? "left" : "right") :

                    (largestVal < 0 ? "up" : "down")

                  )

                );                   

                Pan(x, y);

                zoomOrPan = true;

              }

              break;

          }

 

          // If not zoom or pan, we check for orbit

 

          if (!zoomOrPan && largestAbs < 100)

          {

            // To determine whether we have to orbit, get the normal

            // to the hand's palm

 

            var handNorm = hand.PalmNormal;

 

            if (Math.Abs(handNorm.Roll) > 0.4)

            {

              // Orbit left or right when there is "roll"

 

              WriteLine(

                "Orbit " + (handNorm.Roll < 0 ? "right" : "left")

              );

              var zAxis =

                _ed.CurrentUserCoordinateSystem.CoordinateSystem3d.

                  Zaxis;

              Orbit(zAxis, 0.5 * handNorm.Roll * Math.PI / 12);

            }

            else if (handNorm.Pitch < -1.5 || handNorm.Pitch > -1.0)

            {

              // Orbit up or down when there is "pitch"

 

              var pitch =

                handNorm.Pitch < -1.5 ?

                handNorm.Pitch + 1.5 :

                Math.Abs(handNorm.Pitch + 1.0);

              WriteLine(

                "Orbit " + (pitch < 0 ? "up" : "down")

              );

              OrbitVertical(0.5 * pitch * Math.PI / 12);

            }

          }

        }

        else

        {

          if (ptrs.Count > 0)

          {

            // Set the cursor position to be the

            // tip of the primary pointable

 

            var tip = ptrs[0].TipPosition;

            _jig.Position = new Point3d(tip.x, -tip.z, tip.y);

            _jig.ShowCursor = true;

          }

        }

      }

    }

  }

 

  public class NavigationCommands

  {

    [CommandMethod("LEAP")]

    public void LeapMotionNavigation()

    {

      // Cancel out of any existing geometry creation

 

      GeometryCommands.LeapMotionGeometryCreationCancel();

 

      var doc =

        Application.DocumentManager.MdiActiveDocument;

      var db = doc.Database;

      var ed = doc.Editor;

 

      using (var tr = doc.TransactionManager.StartTransaction())

      {

        var ms =

          (BlockTableRecord)tr.GetObject(

            db.CurrentSpaceId, OpenMode.ForWrite

          );

 

        // Create our jig

 

        var lj = new LeapJig(doc);

 

        // Create a camera for our current document to adjust

        // the view

 

        var cam = new Camera(doc);

 

        // Creating a blank form makes sure the SyncContext is

        // set properly for this thread

 

        using (var f1 = new Form1())

        {

          var ctxt = SynchronizationContext.Current;

          try

          {

            if (ctxt == null)

            {

              throw

                new System.Exception(

                  "Current sync context is null."

                );

            }

 

            // Create our navigation listener to receive events

 

            var listener = new NavigationListener(ed, cam, ctxt, lj);

            using (listener)

            {

              if (listener == null)

              {

                throw

                  new System.Exception(

                    "Could not create listener."

                  );

              }

 

              // Use the listener to create the Leap Motion

              // controller

 

              using (var controller = new Controller(listener))

              {

                if (controller == null)

                {

                  throw

                    new System.Exception(

                      "Could not create controller."

                    );

                }

 

                // Run the jig

 

                PromptResult pr;

                do

                {

                  pr = ed.Drag(lj);

                  if (lj.EnterPressed)

                  {

                    if (lj.Drawing)

                    {

                      var ent = lj.GeneratePathEntity();

                      ent.ColorIndex = 1;

                      ms.AppendEntity(ent);

                      tr.AddNewlyCreatedDBObject(ent, true);

                      lj.Vertices.Clear();

                      lj.Drawing = false;

                    }

                    else

                    {

                      cam.Reset();

                    }

                    lj.EnterPressed = false;

                  }

                } while (pr.Status != PromptStatus.Cancel);

              }

            }

 

            tr.Commit();

          }

          catch (System.Exception ex)

          {

            ed.WriteMessage("\nException: {0}", ex.Message);

          }

        }

      }

    }

  }

}

In the below demo video I’ve used a finger rather than a pointing tool (such as a chopstick), as that allows me to switch between model creation and navigation seamlessly. But the code will work very well with a chopstick, too. :-)




That’s it for my current technical investigations into the possibilities for the Leap Motion device. At some point I’d like to see whether some kind of geometry manipulation is possible, but that’s a tougher nut to crack, and for another time.

To wrap up “Leap Week”, I’ll post again tomorrow to summarise my thoughts on this interesting technology and where I find it applicable to our industry.

blog comments powered by Disqus

Feed/Share

10 Random Posts