Visualizing (and popping) spheres is all well and good, but clearly it’d ultimately be much more interesting to visualize more complex objects in an AR scene. The good news is that PointCloud Browser supports loading models from .OBJ files, a format generated by a number of Autodesk products – particularly those used in the Media & Entertainment space. I’m not a big user of 3ds Max or Maya, and so decided to take a shot at using one of our newer, consumer-oriented iOS apps called 123D Creature.
123D Creature is cool (and great value at $2). It provides character modeling and sculpting capabilities to your iPad, and is even useable by 3D modeling luddites such as myself.
You start by using some straightforward tools to manipulate a default skeleton…
… into something with the basic characteristics you’re looking for. At which point you “bake” the skeleton and can start working on it with clay-sculpting and painting tools.
As a first attempt I managed to create this:
Nothing earth-shattering, but I was pretty amazed to have even made something this tolerable on a first pass given my experience and level of artistic talent.
Getting an .OBJ file from a 123D Creature model is very simple: from the Projects menu, you simply select “Export Mesh”, at which point you can choose either to email it to yourself or to send it to iTunes. These meshes are generally pretty big – once you’ve baked the skeleton they generally run to around 25MB at a minimum – so I’ve tended to use iTunes to get at them.
“Sending to iTunes” means a ZIP will show up in the documents associated with the 123D Creature app – from there you can save it locally and extract its contents. The ZIP file contains the .OBJ file as well as any textures that you’ve either painted or applied.
To load an .OBJ file into a PointCloud Browser scene, I used the following very simple HTML page:
The big issue I faced was with the size – in bytes – of the mesh. As mentioned earlier, the size of any baked model – and there are some fantastic examples being created and shared by the user community – starts at around 25MB, which leads to a few problems: firstly, it’s a 25MB resource that needs to be downloaded when loading the scene and, secondly, this also means more resources are required at runtime to display the mesh.
On my iPad 2 it turned out to be prohibitive: I wasn’t able to see any baked meshes inside my AR scene. The level of detail maintained is clearly intended for processes such as 3D printing rather than reality augmentation on mobile devices, so you can certainly understand why these meshes feel a little heavy for our purposes.
The good news is that skeleton meshes – prior to the baking process – are much smaller (typically weighing in between 500KB and 5MB). You don’t have the finer detail you get with a baked model, but then you can still build in a fair amount of interesting detail with just the skeleton-editing tools.
Here’s an example of a somewhat more elaborate skeleton inside 123D Creature…
And when we load it inside a PointCloud Browser scene with the above code we can see it displayed relatively well (albeit translucently and with no material – as that gets applied post-bake, of course).
When iOS devices have the capability to deal with more complex meshes – and it may well be that iPhone 5 and iPad 4 devices already do better than my aging iPad 2 – then this could be pretty interesting way to visualize more complex meshes. And that doesn’t just go for the output from 123D Creature, of course: you could also be generating your own .OBJ output from a number of Autodesk (and other) products.
Now that I’ve got a taste for this, next time I’m going to take a shot at using Project Pinocchio over on Autodesk Labs to generate a character and getting that displayed inside a PointCloud Browser scene.
This week has so far had an AU theme to it, just as last week we talked exclusively aboutLeapMotionandAutoCAD. Perhaps I’m sub-consciously shifting this blog to a weekly-themed format? Hmm.
Like many of you, I’m sure, I received an email over the weekend to let me know that the recorded sessions from Autodesk University 2012 are now available online (for anyone with a valid AU online account).
There is loads of great content up there on the AU Online site. I strongly recommend checking out the content from the other members of my team who presented at AU 2012, for instance, even if these particular sessions were unfortunately also not recorded:
Well-documented approach for creating AutoCAD apps that work with Google App Engine
Please feel free to post comments with links to other sessions that people might find of interest (posting links to your own is fine, too, as long as they’re at least vaguely development-focused). I know the ADN team presented many well-received and valuable sessions at AU 2012, for instance.
After introducing the topic – as well as creating our basic, local web-service – in the last post, today we’re going to publish our MVC 4 Web API application to the cloud and see it working from a number of different client environments.
Preparing to publish to Azure
Now that we’re ready to publish to Azure, we need to add a deployment project to our solution. Right-click “ApollonianPackingWebApi” in the Solution Explorer and select “Add Windows Azure Cloud Service Project”. This will add a new project into our solution.
We can now double-click the entry under the “Roles” folder in order to adjust the parameters for that role.
It’s here that we can adjust the number and of size of the instances to deploy to Azure, as well as more advanced settings related to Virtual Networks and Caching.
Then we can right-click on the newly added project and select “Publish…”.
We need to sign in to MSDN in order to get our credentials – these get downloaded to your local system in a .publish file – and after selecting “Next” we can add a new cloud service in our preferred location, as well as choosing whether to post to staging or production (we’ll be lazy and go straight to production) and specifying remote desktop settings in case we want to connect to the VM instance hosting our role (sometimes needed in case of debugging).
The actual deployment process can take some time (~5 minutes or so), at which point we should see a “completed” message inside Visual Studio:
Now the site will be ready for testing at the URL you assigned to your cloud service, e.g.:
So that’s all there is to it – we now have a functioning, cloud-based web-site and -service.
To get information on the web-service’s status – including its usage, cost & billing information – you can log into the Windows Azure Management Console:
Calling our web-service from anywhere*
* AutoCAD, Android, iOS, WinRT, HTML5 & Unity3D.
In this section, we’ll take a whirlwind tour of some different client environments.
Let’s start by revisiting AutoCAD, looking at some C# code that calls into our web-service rather than the local F# code.
At the core of this implementation we need some HTTP-related code to call the web-service and some JSON-related to code to parse the results.
Here’s the function we’ll use to call the web-service:
privatestaticdynamic ApollonianPackingWs(
Editor ed, double p, int numSteps, bool circles
)
{
string json = null;
// Call our web-service synchronously (this isn't ideal, as
// it blocks the UI thread)
HttpWebRequest request =
WebRequest.Create(
"http://apollonian.cloudapp.net/api/" +
(circles ? "circles" : "spheres") +
"/" + p.ToString() +
"/" + numSteps.ToString()
) asHttpWebRequest;
// Get the response
try
{
using (
HttpWebResponse response =
request.GetResponse() asHttpWebResponse
)
{
// Get the response stream
StreamReader reader =
newStreamReader(response.GetResponseStream());
// Extract our JSON results
json = reader.ReadToEnd();
}
}
catch (System.Exception ex)
{
ed.WriteMessage(
"\nCannot access web-service: {0}", ex.Message
);
}
if (!String.IsNullOrEmpty(json))
{
// Use our dynamic JSON converter to populate/return
// our list of results
var serializer = newJavaScriptSerializer();
serializer.RegisterConverters(
new[] { newDynamicJsonConverter() }
);
// We need to make sure we have enough space for our JSON,
There’s really not a great deal to it, although there’s a little work going on to deserialize the JSON returned. If we were targeting .NET 4.5 rather than 4.0, we could make use of some new capabilities in the .NET Framework to parse JSON, but this version makes use of a 3rd party JSON serializer (from here). One thing I liked about this particular implementation was its use of .NET 4.0’s dynamic keyword to simplify parsing the JSON. This capability has apparently now been added to the Json.NET library, so if I was starting again I might possibly choose that, instead (I’ve used it successfully on other projects calling web-services).
Our main implementation – excluding the code requesting data from the user – now becomes:
Transaction tr =
doc.TransactionManager.StartTransaction();
using (tr)
{
// Start by creating layers for each step/level
Utils.CreateLayers(db, tr);
// We created our Apollonian gasket in the current space,
// for our 3D version we'll make sure it's in modelspace
// The Layer (and therefore the colour) will be based
// on the "level" of each sphere
s.Layer = tup.L.ToString();
btr.AppendEntity(s);
tr.AddNewlyCreatedDBObject(s, true);
}
}
tr.Commit();
ed.WriteMessage(
"\nCreated {0} spheres.", res.Count
);
}
As you’d expect, the code works in a very similar fashion to the previous implementation (it’s the same code doing the work, just in a geographically different location :-).
Supporting multiple platforms
We’ll now take a look at some options for creating viewers of our 3D data on different platforms.
We’ll start by looking at some native clients for Android, iOS and Windows 8, before looking at one cross-platform toolkit (Unity3D) and then HTML5 (via WebGL).
If you’re interested in continuing the discussion, there’s a conveniently timed round-table session following on after this session (hosted by myself and Philippe Leefsma from the ADN team):
The Android stack is Java-based, which made it a very familiar environment (at least with respect to the code created) for a C# developer such as myself. The tooling was a bit different – the IDE I used was Eclipse – but I was surprised to find out how much I ended up enjoying it: there are a few quirks, but there are also capabilities I’d really like to see in Visual Studio, such as warnings regarding unused namespaces.
The 3D object library – to avoid making low-level calls into OpenGL ES 2.0 – was named Rajawali. An open source toolkit developed by Dennis Ippel in the UK, who provided me with some great support (and even custom features) during the implementation. And the results were very impressive.
Apollonian Viewer for iOS
Now I fully admit I have trouble with Objective-C. I understand its syntax is dictated largely by its origins (it was apparently inspired by Smalltalk, back in the day), but I have become so very used to a traditional structure for calling methods (object.method(arg1,arg2)) that I find Objective-C very difficult to adjust to.
That said, it does appear to be a very powerful, capable and well-loved language. It’s just not something In enjoyed working with closely, myself.
The development environment was Xcode, which I found to be convoluted and unstable, but I assume this is something you get used to.
The 3D object library was iSGL3D – I also looked into a few others but found this to be the best fit, overall. Once again it was based on OpenGL ES 2.0, but the similarities to Rajawali ended there: I found it difficult to get good results (perhaps because I didn’t have direct help from the developer and struggled with Objective-C, in general).
Apollonian Viewer for WinRT
Developing a WinRT-focused client – one that works a Windows Store application on Windows 8 – was interesting. I had already done some work on WinRT, but writing a 3D viewer meant diving into DirectX. In theory you’re supposed to call this from C++ but I ended up wimping out and making use of SharpDX to bridge to it from C#.
Development was done in VS2012 (which was just fine), but because – at least at the time of writing – I couldn’t find a decent 3D object library, I had to get down and dirty and write my own pixel and vertex shaders. A pretty painful experience – I had to learn a great deal about rendering pipelines that I’ve since forgotten – but the results were pretty impressive.
Apollonian Viewer for HTML5
In order to understand the possibilities around HTML5 – which many people believe to be the future for cross-platform development – I decided to dive in and create a WebGL-based viewer for our data.
HTML5 means JavaScript – which I like better than Objective-C, although it’s far from being my favorite language – but I did find the tools available to me had evolved considerably since I last used it in earnest: modern browsers have pretty advance debugging capabilities and even Visual Studio does a pretty good job with the language.
I chose Three.js as the 3D object library, and found it excellent: the results were impressive. It worked well from most browsers – although had to fall back to “canvas” rendering in IE, which was a shame. WebGL is hardware accelerated, so the performance was great.
Viewing an apollonian packing in a Unity scene
And finally, let’s take a look at Unity, a popular, cross-platform game engine that's increasingly used for visualization. I had really wanted to try out this toolkit: I’d heard great things about from a number of people and also felt it wouldn’t be fair to only focus on native apps and HTML5 without taking a look at it.
I haven’t written the title of this section as if I had written a new viewer, although some people have used Unity to do just this: I merely wanted to make use of a standard Unity scene and add the results of our web-service call into it.
I used Unity on OS X (and also had a play with the Windows version) and was able to target a number of environments for free (desktop environments and the web). You can pay to be able to target additional platforms – the list is impressive. From a programming perspective it was easy: I used C# – Unity uses Mono to make this work when not on Windows – but might also have chosen JavaScript. The development environment – when I needed to work on code outside the Unity scene editor – was MonoDevelop: a fairly decent IDE.
Summary
To summarize what we’ve seen in this session…
We extracted some F# code from an existing AutoCAD application
We placed it behind a web-service implemented using ASP.NET MVC4
We published the web-service to Windows Azure
We then modified the AutoCAD client to call the web-service
We then saw how we could also use the data from…
Native apps: Android, iOS, WinRT
Web apps: HTML5 & WebGL
Cross-platform apps: Unity
Overall the experience of creating the web-service was straightforward, although admittedly we kept things really simple: if we’d chosen to implement authentication life would have been at least marginally more interesting. Posting to Azure and managing the deployment was also made very easy by the integrated and standalone tools.
It was fun to do some native development, to understand what’s involved (although I enjoyed Android, WinRT and iOS in that order, I would say). On balance, though, I expect HTML5 to come into its own – even on mobile devices – over the coming year, and if you need to support multiple platforms it’s well worth investigating the cross-platform tools that meet your requirements.
Want to continue the discussion? Come along to the round-table in 30 minutes time!
I finally managed to wrap-up my AU material over the weekend. Here’s the first part of the handout for my Windows 8-related session (if you log in with your AU account, you’ll be able to access the accompanying presentation and sample project).
This is an “intermediate” class (rather than “advanced”), so at times it may seem fairly high-level. That said, if anyone sees issues with the material – especially important topics that have been glossed over – please do let me know: I’d be happy to make corrections (especially in advance of the class being delivered :-).
Introduction
Windows 8 has recently been released to a great deal of interest around the globe. It has been branded – and this is in some ways backed up by the $1.5+ billion marketing spend around the launch – as the most ambitious release of Windows ever. It certainly presents a very different UI paradigm to its users – something for which Microsoft has been both praised and criticized, inevitably – while also maintaining a strong focus on running legacy (now termed “desktop”) applications
Let’s start by taking a look at the Windows 8 Start screen:
The screen contains colourful – and actually very informative – tiles that are clearly targeted at touch-centric, mobile devices.
Devices that specifically target the Windows RT sub-system will run on both traditional Windows 8 machines (with Intel processors, which means they can run WinRT apps as well as traditional desktop apps such as AutoCAD) and on the new ARM-based devices such as the Microsoft Surface tablet and Windows Phone 8 handsets.
In this class, we’re going to spend some time looking at Windows 8 – and specifically WinRT – as well as the development challenges and opportunities it presents.
Windows Runtime – The New Sandbox on the Block
Windows Runtime – which as we’ve already seen is commonly abbreviated to WinRT – is Microsoft’s answer to Android and iOS: two application sandboxes that have had explosive growth over the last few years. Windows has its own, highly popular application execution environment – the .NET Framework – but it seems the belief within at least some divisions at Redmond was that this was not the appropriate model for working with next-generation mobile devices. Attempts have been made in the past to use a subset (which typically means a separate, streamlined version) of the .NET Framework in this way (such as the .NET Compact Framework and Microsoft Silverlight), but for better or worse WinRT is Microsoft’s latest attempt at addressing this particular market need.
WinRT is a complete layer on top of the core services belonging to the Windows OS. While with .NET it has often been needed to use the Platform Invoke capability to call through to the underlying Win32 API, with WinRT this is no longer needed or even allowed (at least not if you want to make your application available via the Windows Store).
So what does it mean, this idea of a sandbox? It’s basically a controlled execution environment for software: one that only provides a restricted subset of OS capabilities to applications. By not providing “risky” capabilities, sandboxes allow untrusted code to be executed in a controlled way: the ability to access certain resources – such as a user’s documents – must be specifically requested by the app, which get transferred to privileges that must be granted specifically by the user.
Aside from providing a much more secure and trustworthy environment for applications to execute, WinRT also provides the ability for code to be written in a variety of languages. We’ll look at that in more detail shortly.
WinRT – How it looks different
The first thing that strikes you when you see Windows 8 is that it looks very different. As mentioned earlier, the Start screen presents some highly colourful tiles to the user, some of which are “live” and present information deemed important by the apps to whom they belong.
The original name for the design language used for Windows 8 – as well as Zune, Xbox and Windows Phone – was Metro. In fact that’s the reason this class is named as it is. A few months ago, however, Microsoft stated the name had actually been a codename and went through a significant re-branding exercise (as despite being a codename, the Metro moniker had proved popular and was used fairly pervasively to describe the design language and the apps that make use of it). It’s believed this change of approach was driven by legal action from Metro AG, but that’s not really important for us to focus on.
Microsoft adopted the term “modern UI style apps” before finally settling – fairly recently – on the term “Windows Store” apps. Whenever you see the title of this class – which has clearly now been set in fairly strong concrete – please translate it automatically to “Developing a simple Windows Store application for Windows 8”. Windows Store apps are full-screen, chromeless and are clearly targeted at touch-centric devices (although it’s always possible to use them on devices that are not touch-capable).
There were various influences that drove the look of what was formerly known as “Metro”. While not a core part of this class, it’s interest to note them:
In addition to these inspirations, the “Metro” design language is founded on some core principles:
Pride in craftsmanship
Fast and fluid
Authentically digital
Do more with less
Win as one
One point that’s worth calling out specifically is “authentically digital”: while iOS has (at least until now) taken a firm position in favour of skeuomorphism (in this context the act of imitating the real world in the digital one), Microsoft has chosen another path, avoiding the temptation of making digital imitations of real-world objects such as bookshelves. The message here is: don’t be afraid of digital versions being different, as those differences are also what make them valuable.
For more detailed information on these topics, I recommend these twopresentations (the second of which was the source of the included images, many thanks to Laurent Bugnion).
WinRT – How it is different
The programming model and the tools you need to use to make use of it are – on the one hand – familiar to .NET developers, but there are significant differences. There’s a lot to learn with WinRT.
Here are four main areas of difference, that we’ll cover in some detail:
Different object model for OS capabilities
Asynchronous operation enforced for anything but “instant” responses
Strict control over disk access
Application interop only via contracts or file associations
The WinRT Object Model
When you first start working with WinRT from C#, there’s a great deal that’s familiar (especially if you’ve been working either with WPF or with Silverlight). There is a relationship with .NET, but it’s not actually the central mechanism that’s used to connect with the core OS layer. This happens to be COM – somewhat curiously for those of us that have been happy to distance ourselves from using this technology over the .NET years – but that’s really an implementation detail: you’re unlikely to have to get down and dirty with IUnknown (unless you really want to, of course).
WinRT and the CLR have distinct type systems that are effectively merged by the WinRT metadata adapter. At times it will seem a little strange to be working with the different types in the same application: for instance, some events are exposed via the CLR while others come from WinRT. You may notice the difference from the names (and types) of the arguments that are assigned automatically, but mostly these differences don’t get in the way at all.
When using C# or VB – and, inevitably, XAML for the UI – you’ll be targeting a particular .NET Framework profile (the .NETCore implementation). This provides access to the approved subset of the CLR for use with WinRT. At runtime, the full CLR is actually loaded and theoretically available to applications but the reality is that an application won’t be able to make use of it: while it’s possible for developers to hack their way beyond this approved subset of the Windows API, the Windows Store vetting process will exclude any apps submitted for posting that use techniques such as Platform Invoke or network loopback to play outside of the sandbox.
Asynchronous operations are now the norm
.NET developers have been playing with the async capabilities released in .NET 4.5 for some time, but with WinRT this mechanism takes on a whole new significance: any operations that could in theory take more than 50ms to complete are only made available via asynchronous API calls.
The intention is clearly to stop long-running API calls from blocking the UI thread and resulting in apps “whiting out”. Asynchronous calls help keep the Windows 8 UI responsive and fluid.
And the good news is that the async and await keywords make this really easy to deal with from your code, for instance:
In this situation, the await keyword tells the compiler to create an event that gets fired to continue execution of the application’s code once the results of the GetFileAsync() call are returned. The pain previously associated with performing operations asynchronously is now thankfully abstracted away by the compiler.
Restricted access to the file system
Applications making use of the WinRT object model no longer have arbitrary access to the file system. This is clearly intended to make applications more easily trusted by their users.
There are a relatively small number of locations that applications can request access to – they need to do so by modifying the Package.appmanifest to add “Capabilities”. Applications can – for instance – use this mechanism to request read/write access to the current user’s Documents, Pictures, Videos and to Removable Storage devices.
In addition to asking for access to files in this very restricted set of locations, applications also need to register themselves as associated with the types of the files they wish to access.
All these permissions get presented to the user when installing the app – and via the application’s Settings information – to make it very clear the type of information the app intends to access.
When an app asks the user to select a particular file from the system more freedom is granted to the app: it is effectively implied by the selection that the app is being granted permission to work with a particular file.
Application interop
Applications have very limited abilities to interoperate in the WinRT world: even if the system’s underpinnings are COM – a mechanism that was intended to facilitate the passing of information between software components – WinRT has been deliberately limited in this very important respect. This imposes good behavior on app developers, but many will find it restrictive: there will certainly be times when apps want to pass information between each other without it being a user-initiated operation. But for the good of the user experience, this is the choice that has been made by the implementers of WinRT.
As mentioned previously, access to the file system has been deliberately limited, but it is still possible to use it to share data at a user level. The Registry, on the other hand, is not accessible from WinRT apps, and so can no longer be used to pass information between applications.
There was a brief reference to “network loopback” in a previous section: this is the act of hosting a lightweight internet server on the local host (127.0.0.1) and using that from another WinRT app to exchange information. This is a fairly common interop technique, but this capability has been at least partially blocked for apps. And those that do manage to implement this technique will not make it through the Windows Store vetting process.
So what options do apps have for sharing information?
There’s one very specific “contract” (which is basically a protocol that needs to be implemented and registered with the framework) that apps can implement in order to share information at the request of the user (the user can choose to pass data from one application to another via the Share charm). It’s in this way that apps can have user-initiated interop – such as a user choosing to share a photo with the Mail app.
It’s also possible for WinRT apps to launch a desktop app that has been associated with a particular file-type. We’ll see this later on, where we choose to launch AutoCAD from our companion app to load a particular DWG file.
Some additional points about WinRT apps
The WinRT sub-system – while it runs on a traditional Windows 8 PC – is really intended for lightweight, touch-centric apps that will be run on a mobile device of some kind. WinRT apps are 32-bit only and are also not expected to consume significant quantities of memory: any that do will be jumped on by the WinRT runtime.
In fact the runtime is being left to manage application memory on a fairly general basis: apps typically don’t have explicit close or exit buttons (although you could, of course, choose to implement one) as the runtime is expected to terminate and resurrect apps as it needs to. It is possible to close applications via the task manager – or to get the app list on the left, which in recent versions of the OS provides a close option for each – but the expectation is that most users will not want to prune the list of running apps manually.
It’s generally not anticipated that software products such as AutoCAD will get ported to the WinRT sub-system: Microsoft doesn’t expect that heavyweight apps will get ported to WinRT. That’s not to say that it’ll never happen – or that AutoCAD components such as RealDWG will never be made available for WinRT – but there are significant architectural questions to resolve beforehand. For instance, with restricted file system access, how would AutoCAD handle accessing and loading support files or external references?
So it’s clear that we’re not going to be talking about writing an AutoCAD-hosted WinRT app in this session – because without WinRT-based AutoCAD, there is no WinRT API for it. Let’s then take a look at what application possibilities there might be, especially considering that application interop is so limited between WinRT and desktop apps.
In the next part, we’ll take a look at the opportunities for Windows Store apps to complement desktop software such as AutoCAD.
Update:
I found this article an interesting read, today: it ties the vision around WinRT together with the subject of my other AU session, moving desktop functionality to the cloud via Windows Azure.
As we reach the end of this long series of posts on moving code to the cloud – and a look at ways to use the functionality from a wide variety of applications, many of them on mobile devices – I felt it was worth putting together a quick summary post to reinforce the overall message (which may have been a bit lost in the sheer volume of information).
Firstly, here are the posts in this series, including those looking at the original creation of the “desktop” application functionality:
Here are the client application projects I’ve posted for Android, iOS & WinRT.
Perhaps it’s time to say a few words about the experience I had with each of these environments…
Unity3D was great and would certainly allow you to create a 3D app that could be used in any of the other environments (with the possible exception of WinRT for now, at least). I’d definitely consider this again to create a cross-platform 3D app.
Android surprised me: I really liked the language (Java), the chosen 3D framework (Rajawali) and the development tool (Eclipse).
WebGL & Three.js proved to be both easy to use and extremely powerful. Definitely an area to watch as WebGL gets more browser support, especially on mobile platforms.
iOS was very strange for me, but didn’t prove to be so awful to develop for: while I found the Objective-C syntax to be quite jarring and the Xcode tool to be both confusing and somewhat unstable, the capabilities of the platform and its primary language were quite mature: you can do quite a lot with relatively few lines of (admittedly pretty weird-looking ;-) code. I found iSGL3D reasonably good, although I was ultimately disappointed with the results I was personally able to get from it.
WinRT was fairly familiar – given my history with .NET – although the lack of a higher-level framework (with a simple sphere primitive, for instance) meant I struggled more with this than I’d have thought. I hope/expect that to be addressed by somebody at some point: while SharpDX allowed me to at least use C# to create a Metro-style Direct3D app, I really would have benefited from a higher abstraction layer to code against.
And to conclude…
This series was really intended to demonstrate that there are significant benefits in making core algorithms cloud-based, not only from an architectural perspective, but from the broad range of applications it enables you to provide to your customers in the long term. And also that this technology really isn’t hard: most of the client apps took a day or two to develop and none took more than a week.
I’ve chosen to deal with graphical data in this series of posts – mainly to make them more visually interesting – but it’s arguably even more compelling to broaden the reach of associated, non-graphical design data using this kind of approach.
Hopefully some of you have followed along with these posts, and derive at least some benefit from seeing similar code working in a variety of languages on a variety of technology stacks. It’s certainly been a fun one to work on: diving so deeply into the respective technologies has at times been challenging but highly educational. :-)
After starting the sub-seriesfocused on iOS, I held off completing it until I could actually test the code on a physical iOS device. A big thanks to Andy Chang from our Toronto office for getting me set up with the ADP membership and my iPad 2 added to the list of usable development devices.
I won’t talk about the steps needed to provision apps for iOS devices – there seems to be enough information available on the web for that – but I will say it ended up being less complicated than I expected. That’s not to say it’s easy to package things up for posting to the App Store – I haven’t gone through that, myself – but just deploying the app to a physical device for testing was reasonably straightforward.
That said, there was some non-trivial work needed to get an app working on both the iOS simulator and a physical device, mainly due to our use of the iSGL3D. The issue came, in this case, from the fact that iSGL3D has a problem with ARC (Automatic Reference Counting). The path of least resistance ended up being to build it into a universal, static library, which could then be used to build an iOS app that targets both the iOS Simulator (which is i386-based, as it works on the Mac) and the physical, ARM-based iOS device.
I’m still not very happy with the lighting I was able to get working using iSGL3D: the “out-of-the-box” lighting capabilities – presumably built into the base material types – were much nicer with Three.js or Rajawali. I did my best by reading up on iSGL3D lighting, OpenGL lighting and even going back to basics on diffuse, ambient and specular lighting, but the results weren’t as good as I’d have liked. This is probably down to me, rather than the iSGL3D framework, per se, but still.
Here’s the latest version in action on the iOS simulator (which turns out to be comparable to the physical device, graphics-wise, even if the framerate is indeed much lower):
That's it for the sub-series on iOS (for now, at least) and for the overall series on cloud & mobile (from a technical perspective). In the next post, we’ll take a look back at the series and comment on the journey we’ve been on for the last month or so.
In the last post, we saw some code to implement a simple 3D viewer of data coming from our Apollonian web-service on iOS. In this post, we’ll add support for touch gestures, as well as a simple message box announcing when the web-service is unavailable.
When compared with Android, iOS provides much higher-level gesture information via its UIKit framework: you basically get callbacks to indicate when the screen has been tapped or swiped, or when pinch or rotate gestures have been performed. This is all very helpful, in the sense that you don’t have to do so much low-level running around in your code – and risk that different apps respond to touch in a different way – but it does come at the expense of flexibility. For instance, if you want to support swipe gestures, you can really only do so in the four main directions (up, down, left, right) and you need a separate recognizer object for at least vertical and horizontal directions, and quite possibly all four.
In the Android version of this app, we had to decide how swipe gestures would be implemented at a fairly low level, but the upside was that we could calculate a more precise direction and use that to adjust the spin of our model. Now I’m sure that it’s possible with iOS to hook into events at a lower level, to essentially do the same as with Android, but there are a couple of reasons I’ve decided not to. The first is that I don’t really need feature parity between the two viewers – as this is mostly research-related – but more importantly I have doubts that I’ll have the same ability to rotate our model in an arbitrary direction, as Dennis Ippel kindly enabled for us in his Rajawali framework (I haven’t seen the same capabilities, necessarily, in iSGL3D, and have less hope of getting them implemented). I’d ideally like to specify a rotation around an axis that’s perpendicular to the swipe direction – perhaps using a quaternion – and then apply that additively to the model.
Again, it’s probably something that I could implement myself, given enough time and effort, but I’ve decided to keep things simple. The addition of the rotate gesture – even if it gets reset when the model starts to spin – seems a reasonable complement to the existing behaviour of spinning in one of four directions, at least.
Before we look at the updated code, here’s a quick video of the application in action on the iOS Simulator:
And with that, on to the code. Here’s the updated Objective-C implementation of our Apollonian Viewer. As you’ll have seen in the video, I’ve gone ahead and added a splash-screen and icons to the project, even if we don’t have much else by way of a UI, still.
I’ve managed to get into the Apple Developer Program via an Autodesk subscription, but there are some delays with me being able to test the app directly on a physical device. I’m still waiting to do so before I determine what additional UI needs to be implemented (in terms of progress bars, particularly), although I could probably go ahead and add some level-selection capability, at least. I may just move on to other things and revisit this at some point in the future – we’ll see.
Last week, it was allaboutAndroid. This week, I’ve started taking the plunge into the world of iOS. I’ve been using a Mac for some time – mainly to wean myself away from being so Windows-centric, but also with a view to working more with AutoCAD for Mac from a development perspective – but this was the first time I’d actually forced myself to write anything for either OS X or iOS.
It all came as a bit of a shock, initially, even though I was generally aware of the strangeness of Objective-C with respect to its message-passing syntax. So while I enjoy learningdifferentprogramminglanguages, I found I really struggled with Objective-C. But anyway – obviously lots of people have managed to get their heads around it (and many of this blog’s readers will have done so, I’m sure), so at least there is a fair amount of help available out there on the web.
Someone has already commented on the fact that you can use C# to build apps for iOS and Android directly – such as with a toolkit like Xamarin or an engine like Unity3D – but the point of this series of posts is as much about driving my own learning as it is about presenting my readers with easy options (sorry for being selfish, but that’s just how it is). And I think there’s value in seeing the “native” approach across a variety of platforms – while knowing that options exist allowing you to maintain a largely platform-independent codebase to target them.
And so on to my deep-ish dive into iOS…
My first challenge was identifying a decent (and free) 3D engine for our viewer – all of which seem to be based on OpenGL ES, much the same as for Android. I started by looking at Cocos3D (which is based on the apparently very popular Cocos2D), but ended up discarding it as it didn’t appear to have a sphere primitive available (which was a bit of a deal-breaker for me, given the problem space ;-).
I moved on to look at iSGL3D, which certainly appeared to provide what I was looking for from a 3D engine. I spent some time looking at its online tutorials, which were reasonably comprehensive, before trying the “tests” provided with the framework and building my first basic app with the Xcode 4 template. I went through a little unnecessary thrashing, as I pulled down the latest file versions directly from GitHub before realising I really needed to install the latest stable build (version 1.2.3 at the time of writing).
But, that aside, the process was reasonably straightforward. I modified the contents of the “Hello World” files created by the Xcode template (renaming them, too, of course), to be as follows…
It’s worth noting that – in an effort to make the code a more familiar and consistent with the other code I post here – I’ve thrown away the book on Objective-C coding conventions (much as I did for Java, last week). That’s partly for the benefit of this blog’s readers, but also for my own sanity. ;-)
Firstly, the ApollonianViewer.h header file:
#import "isgl3d.h"
@interface ApollonianViewer : Isgl3dBasic3DView
{
@private
NSMutableArray * _materials;
Isgl3dNode * _container;
Isgl3dSphere * _sphereMesh;
}
-(void)createSphere
:(double)radius
x:(double)x y:(double)y z:(double)z
level:(int)level;
@end
And now the main ApollonianViewer.m implementation file:
#import "ApollonianViewer.h"
@implementation ApollonianViewer
// Our data member for the received data
NSMutableData * _receivedData = NULL;
// A response has been received from our web-service call
- (void)connection:(NSURLConnection *)connection
didReceiveResponse:(NSURLResponse *)response
{
// Initialise our member variable receiving data
if (_receivedData == NULL)
_receivedData = [[NSMutableDataalloc] init];
else
[_receivedDatasetLength:0];
}
// Data has been received from our web-service call
// Create a directional white light and add it to the scene
Isgl3dLight * light =
[Isgl3dLight
lightWithHexColor:@"A0A0A0"
diffuseColor:@"E9E9E9"
specularColor:@"C0C0C0"
attenuation:0
];
light.lightType = DirectionalLight;
light.position = iv3(4, 0, 8);
[light setDirection:1y:2z:-5];
[self.sceneaddChild:light];
// Set the scene ambient color
[selfsetSceneAmbient:@"000000"];
}
returnself;
}
// Create a single sphere at the desired position with
// the desired radius and level
- (void)createSphere
:(double)radius
x:(double)x y:(double)y z:(double)z
level:(int)level
{
// Create the sphere based on our single mesh
Isgl3dMeshNode * sphere =
[_container
createNodeWithMesh:_sphereMesh
andMaterial:[_materialsobjectAtIndex:level]
];
// Position and scale it
sphere.position = iv3(x, y, z);
[sphere setScale:radius];
}
- (void) dealloc
{
// Make sure we release our materials and sphere mesh
[_materialsrelease];
[_sphereMeshrelease];
[superdealloc];
}
- (void) tick:(float)dt
{
// Rotate around the y axis
_container.rotationY += 2;
}
@end
The app currently does a fair amount less that its Android counterpart – I haven’t implemented any kind of UI, including progress bars, touch gestures, etc. – but there were actually some things that just worked more smoothly: rather than worrying about threading issues, the call to the web-service seemed to execute asynchronously by default, and the code adding spheres worked well, once I’d determined I needed to control the lifetime of my supporting objects rather than allowing them to be garbage-collected at the whim of the iOS runtime. It’s not clear to me how much of this is down to the iSGL3D runtime vs. Objective-C/iOS, but I was pleasantly surprised, either way.
I expect I’ll hit more significant challenges, further down the line, but my initial impression is that the above code is actually impressively functional for the amount there is: it was pretty simple to access a REST web-service and decode the JSON results, for instance, and the iSGL3D coding was also relatively straightforward.
And while looking at the syntax still gives me a headache, at least my nose has stopped bleeding. ;-)
Here’s a screenshot of the app working on the iPad 5.1 Simulator:
I’d really like to see this working on the iPad itself, as before I start tweaking the lighting, etc. to get better results, I’d like to see what, if anything, is due to the lack of GPU-accelerated graphics in the simulator (assuming that’s the case). It seems that to do so I’ll need to sign up for the iOS Developer Progrram at $99 per year, which I find a little annoying but to some degree understandable.
In fairness, the Android simulator can’t even run OpenGL ES 2.0 code, at the time of writing, so being forced to pay to test on a physical device would probably have raised the barrier of entry high enough to put me off working with Android completely. At least there is some option for getting started for free on iOS.
Aside from testing on a physical device, I also want to implement some kind of rudimentary UI – much as I did for Android – so I’ll be working on that before I end up posting the full project.
Subscribe via RSS
Recent Comments
Archives
More...