After introducing Leap Motion and seeing some code to make view changes inside AutoCAD, now it’s time to start thinking about geometry creation.
We’ll see two different approaches to compare and contrast. The first – covered in today’s post – is a very generic integration at the Windows Message-level: as the user’s hand hovers over the device, a component inside AutoCAD translates this into cursor movements. Quick and dirty, but hey – it’s in the second approach (probably in the next post) that we’ll see an ultimately more compelling, higher-level integration.
With the first approach the user will see the cursor move independently from the view direction, which is somewhat at odds with the approach taken by the model navigation UI we saw last time. But this is, after all, the quicker & dirtier of the two approaches and comes with the advantage of it being useable with any AutoCAD command (and ultimately with any Windows app).
That’s the cursor movement covered, but to really deliver a “mouse replacement” we also need to simulate left-button clicks (right-clicks aren’t a priority for this simple prototype). We’ll check the velocity of the individual fingers being tracked by the device: if any are significantly quicker than the velocity of the palm, then we consider that to indicate a click event (which we then simulate inside AutoCAD).
One additional trick is to stop multiple clicks from being sent from subsequent frames: the code adds a timer to essentially prevent any clicks from being sent for one second after the previous one (presumably this is enough time for the movement to complete as a certain finger velocity is needed, after all).
In this way we can basically launch commands – ideally from a toolbar, as the cursor tends to disappear when moved programmatically over AutoCAD’s UI elements and it’s easier for the user to track the cursor’s progress visually over toolbars than the ribbon – and then interact with them inside AutoCAD.
Now I really don’t expect this level of integration to be compelling to users: if all the Leap Motion controller is used for is to replace a mouse then I would consider that a product failure: this is really just the “low hanging fruit”… it’s the fruit higher up the tree – which we’ll start to see in the next post – that are ultimately more interesting. :-)
Below is the C# source that can be added to the the code shown previously to complement it. This code defines two commands but neither is actually needed to run the app: the function implementing the LEAPGEOM command is executed when the app is loaded and the function behind LEAPGEOMX is called when AutoCAD exits.
Adding a call to GeometryCommands.LeapMotionGeometryCreationCancel() within the LEAP command’s implementation (from the last post) would be a very good idea if sitting in the same project (otherwise we’d be attaching two listeners at the same time).
using System;
using System.Threading;
using System.Runtime.InteropServices;
using Autodesk.AutoCAD.ApplicationServices.Core;
using Autodesk.AutoCAD.EditorInput;
using Autodesk.AutoCAD.Geometry;
using Autodesk.AutoCAD.Runtime;
using Leap;
[assembly:
ExtensionApplication(typeof(LeapMotionIntegration.Initialization))
]
namespace LeapMotionIntegration
{
public class Initialization : IExtensionApplication
{
public void Initialize()
{
Editor ed =
Application.DocumentManager.MdiActiveDocument.Editor;
ed.WriteMessage("Leap Motion integration loaded.");
GeometryCommands.LeapMotionGeometryCreation();
}
public void Terminate()
{
GeometryCommands.LeapMotionGeometryCreationCancel();
}
}
public class GeometryCreationListener : Listener
{
[DllImport(
"user32.dll",
CharSet=CharSet.Auto,
CallingConvention=CallingConvention.StdCall
)
]
public static extern void mouse_event(
uint dwFlags, uint dx, uint dy,
uint cButtons, uint dwExtraInfo
);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
private Editor _ed;
private SynchronizationContext _ctxt;
private bool _handling = false;
public GeometryCreationListener(
Editor ed, SynchronizationContext ctxt
)
{
_ed = ed;
_ctxt = ctxt;
}
public override void OnFrame(Controller controller)
{
// Get the most recent frame
var frame = controller.Frame();
var hands = frame.Hands;
var numHands = hands.Count;
// Only proceed if we have at least one hand
if (numHands >= 1)
{
// Get the first hand and its velocity to check for
// zoom or pan
var hand = hands[0];
var handVel = hand.PalmVelocity;
if (handVel == null)
handVel = new Vector(0, 0, 0);
// Check if the hand has any fingers
var fingers = hand.Fingers;
// Only proceed if we see at least two fingers detected
if (fingers.Count > 2)
{
var pos = System.Windows.Forms.Cursor.Position;
var x = pos.X + (int)handVel.x / 7;
var y = pos.Y + (int)handVel.z / 7;
// Set the cursor position
System.Windows.Forms.Cursor.Position =
new System.Drawing.Point(x, y);
if (!_handling && Math.Abs(handVel.y) < 30)
{
foreach (var finger in fingers)
{
// If at least one finger has velocity,
// simulate a mouse-click
if (Math.Abs(finger.TipVelocity.y) > 150)
{
mouse_event(
MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP,
(uint)x, (uint)y, 0, 0
);
// Make sure no further clicks get sent for the
// specified number of seconds
Handle(1);
break;
}
}
}
// Process Windows messages at the end of each frame
System.Windows.Forms.Application.DoEvents();
}
}
}
private void Handle(int secs)
{
// Only handle an event if one isn't already in progress
if (!_handling)
{
// Set the flag to stop other events being handled for
// the specified duration
_handling = true;
// Set a timer to unset the flag once the duration
// has passed
var timer = new System.Windows.Forms.Timer()
{
Interval = (int)(secs * 1000),
Enabled = true
};
timer.Tick +=
(s, e) =>
{
_handling = false;
timer.Stop();
timer.Dispose();
timer = null;
};
}
}
}
public class GeometryCommands
{
private static GeometryCreationListener _listener = null;
private static Controller _controller = null;
[CommandMethod("LEAPGEOM")]
public static void LeapMotionGeometryCreation()
{
var doc =
Application.DocumentManager.MdiActiveDocument;
var db = doc.Database;
var ed = doc.Editor;
if (_listener == null || _controller == null)
{
// Creating a blank form makes sure the SyncContext is
// set properly for this thread
if (SynchronizationContext.Current == null)
{
using(var f1 = new Form1()){}
}
var ctxt = SynchronizationContext.Current;
try
{
if (ctxt == null)
{
ed.WriteMessage(
"\nCurrent sync context is null."
);
return;
}
if (_listener == null)
{
_listener = new GeometryCreationListener(ed, ctxt);
if (_listener == null)
{
ed.WriteMessage("\nCould not create listener.");
return;
}
if (_controller == null)
{
_controller = new Controller(_listener);
if (_controller == null)
{
ed.WriteMessage("\nCould not create controller.");
return;
}
}
}
}
catch (System.Exception ex)
{
ed.WriteMessage("\nException: {0}", ex.Message);
}
}
}
[CommandMethod("LEAPGEOMX")]
public static void LeapMotionGeometryCreationCancel()
{
if (_controller != null)
{
_controller.Dispose();
_controller = null;
}
if (_listener != null)
{
_listener.Dispose();
_listener = null;
}
}
}
}
Here’s the code in action – the video is the same as yesterday’s, but this time we start at the beginning to see how the technique might be used to call AutoCAD commands:
Next time we’ll adapt a technique previous seen in my Kinect investigations to create 3D splines and polylines based on input from the Leap Motion controller.
