December 2014

Sun Mon Tue Wed Thu Fri Sat
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      










« Integrating Leap Motion and AutoCAD: Basic Navigation | Main | Integrating Leap Motion and AutoCAD: 3D Geometry »

January 15, 2013

Integrating Leap Motion and AutoCAD: Command Interaction

After introducing Leap Motion and seeing some code to make view changes inside AutoCAD, now it’s time to start thinking about geometry creation.

We’ll see two different approaches to compare and contrast. The first – covered in today’s post – is a very generic integration at the Windows Message-level: as the user’s hand hovers over the device, a component inside AutoCAD translates this into cursor movements. Quick and dirty, but hey – it’s in the second approach (probably in the next post) that we’ll see an ultimately more compelling, higher-level integration.

With the first approach the user will see the cursor move independently from the view direction, which is somewhat at odds with the approach taken by the model navigation UI we saw last time. But this is, after all, the quicker & dirtier of the two approaches and comes with the advantage of it being useable with any AutoCAD command (and ultimately with any Windows app).

That’s the cursor movement covered, but to really deliver a “mouse replacement” we also need to simulate left-button clicks (right-clicks aren’t a priority for this simple prototype). We’ll check the velocity of the individual fingers being tracked by the device: if any are significantly quicker than the velocity of the palm, then we consider that to indicate a click event (which we then simulate inside AutoCAD).

One additional trick is to stop multiple clicks from being sent from subsequent frames: the code adds a timer to essentially prevent any clicks from being sent for one second after the previous one (presumably this is enough time for the movement to complete as a certain finger velocity is needed, after all).

In this way we can basically launch commands – ideally from a toolbar, as the cursor tends to disappear when moved programmatically over AutoCAD’s UI elements and it’s easier for the user to track the cursor’s progress visually over toolbars than the ribbon – and then interact with them inside AutoCAD.

Now I really don’t expect this level of integration to be compelling to users: if all the Leap Motion controller is used for is to replace a mouse then I would consider that a product failure: this is really just the “low hanging fruit”… it’s the fruit higher up the tree – which we’ll start to see in the next post – that are ultimately more interesting. :-)

Below is the C# source that can be added to the the code shown previously to complement it. This code defines two commands but neither is actually needed to run the app: the function implementing the LEAPGEOM command is executed when the app is loaded and the function behind LEAPGEOMX is called when AutoCAD exits.

Adding a call to GeometryCommands.LeapMotionGeometryCreationCancel() within the LEAP command’s implementation (from the last post) would be a very good idea if sitting in the same project (otherwise we’d be attaching two listeners at the same time).

using System;

using System.Threading;

using System.Runtime.InteropServices;

using Autodesk.AutoCAD.ApplicationServices.Core;

using Autodesk.AutoCAD.EditorInput;

using Autodesk.AutoCAD.Geometry;

using Autodesk.AutoCAD.Runtime;

using Leap;

 

[assembly:

  ExtensionApplication(typeof(LeapMotionIntegration.Initialization))

]

 

namespace LeapMotionIntegration

{

  public class Initialization : IExtensionApplication

  {

    public void Initialize()

    {

      Editor ed =

        Application.DocumentManager.MdiActiveDocument.Editor;

      ed.WriteMessage("Leap Motion integration loaded.");

 

      GeometryCommands.LeapMotionGeometryCreation();

    }

 

    public void Terminate()

    {

      GeometryCommands.LeapMotionGeometryCreationCancel();

    }

  }

 

  public class GeometryCreationListener : Listener

  {

    [DllImport(

      "user32.dll",

      CharSet=CharSet.Auto,

      CallingConvention=CallingConvention.StdCall

      )

    ]

    public static extern void mouse_event(

      uint dwFlags, uint dx, uint dy,

      uint cButtons, uint dwExtraInfo

    );

 

    private const int MOUSEEVENTF_LEFTDOWN = 0x02;

    private const int MOUSEEVENTF_LEFTUP = 0x04;

    private const int MOUSEEVENTF_RIGHTDOWN = 0x08;

    private const int MOUSEEVENTF_RIGHTUP = 0x10;

 

    private Editor _ed;

    private SynchronizationContext _ctxt;

    private bool _handling = false;

 

    public GeometryCreationListener(

      Editor ed, SynchronizationContext ctxt

    )

    {

      _ed = ed;

      _ctxt = ctxt;

    }

 

    public override void OnFrame(Controller controller)

    {

      // Get the most recent frame

 

      var frame = controller.Frame();

      var hands = frame.Hands;

      var numHands = hands.Count;

 

      // Only proceed if we have at least one hand

 

      if (numHands >= 1)

      {

        // Get the first hand and its velocity to check for

        // zoom or pan

 

        var hand = hands[0];

        var handVel = hand.PalmVelocity;

        if (handVel == null)

          handVel = new Vector(0, 0, 0);

 

        // Check if the hand has any fingers

 

        var fingers = hand.Fingers;

 

        // Only proceed if we see at least two fingers detected

 

        if (fingers.Count > 2)

        {

          var pos = System.Windows.Forms.Cursor.Position;

          var x = pos.X + (int)handVel.x / 7;

          var y = pos.Y + (int)handVel.z / 7;

 

          // Set the cursor position

 

          System.Windows.Forms.Cursor.Position =

            new System.Drawing.Point(x, y);

 

          if (!_handling && Math.Abs(handVel.y) < 30)

          {

            foreach (var finger in fingers)

            {

              // If at least one finger has velocity,

              // simulate a mouse-click

 

              if (Math.Abs(finger.TipVelocity.y) > 150)

              {

                mouse_event(

                  MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP,

                  (uint)x, (uint)y, 0, 0

                );

 

                // Make sure no further clicks get sent for the

                // specified number of seconds

 

                Handle(1);

                break;

              }

            }

          }

 

          // Process Windows messages at the end of each frame

 

          System.Windows.Forms.Application.DoEvents();

        }

      }

    }

 

    private void Handle(int secs)

    {

      // Only handle an event if one isn't already in progress

 

      if (!_handling)

      {

        // Set the flag to stop other events being handled for

        // the specified duration

 

        _handling = true;

 

        // Set a timer to unset the flag once the duration

        // has passed

 

        var timer = new System.Windows.Forms.Timer()

        {

          Interval = (int)(secs * 1000),

          Enabled = true

        };

 

        timer.Tick +=

          (s, e) =>

          {

            _handling = false;

            timer.Stop();

            timer.Dispose();

            timer = null;

          };

      }

    }

  }

 

  public class GeometryCommands

  {

    private static  GeometryCreationListener _listener = null;

    private static Controller _controller = null;

 

    [CommandMethod("LEAPGEOM")]

    public static void LeapMotionGeometryCreation()

    {

      var doc =

        Application.DocumentManager.MdiActiveDocument;

      var db = doc.Database;

      var ed = doc.Editor;

 

      if (_listener == null || _controller == null)

      {

        // Creating a blank form makes sure the SyncContext is

        // set properly for this thread

 

        if (SynchronizationContext.Current == null)

        {     

          using(var f1 = new Form1()){}

        }

 

        var ctxt = SynchronizationContext.Current;

 

        try

        {

          if (ctxt == null)

          {

            ed.WriteMessage(

              "\nCurrent sync context is null."

            );

            return;

          }

 

          if (_listener == null)

          {

            _listener = new GeometryCreationListener(ed, ctxt);

            if (_listener == null)

            {

              ed.WriteMessage("\nCould not create listener.");

              return;

            }

 

            if (_controller == null)

            {

              _controller = new Controller(_listener);

              if (_controller == null)

              {

                ed.WriteMessage("\nCould not create controller.");

                return;

              }

            }

          }

        }

        catch (System.Exception ex)

        {

          ed.WriteMessage("\nException: {0}", ex.Message);

        }

      }

    }

 

    [CommandMethod("LEAPGEOMX")]

    public static void LeapMotionGeometryCreationCancel()

    {

      if (_controller != null)

      {

        _controller.Dispose();

        _controller = null;

      }

      if (_listener != null)

      {

        _listener.Dispose();

        _listener = null;

      }

    }

  }

}

Here’s the code in action – the video is the same as yesterday’s, but this time we start at the beginning to see how the technique might be used to call AutoCAD commands:


 

Next time we’ll adapt a technique previous seen in my Kinect investigations to create 3D splines and polylines based on input from the Leap Motion controller.

blog comments powered by Disqus

Feed/Share

10 Random Posts