Making a New Amazon Skill for The Echo

I was prompted to dive back into the Amazon Alexa Skill development by the new Alexa Hackster.io contest.

I quickly classified the development of Alexa Skills into two major categories. They don’t break them down this way, but it makes more sense to think of their development by the requirements of the developer and setup.

  1. Not user specific – Generic information and interactions available to everyone
  2. User specific – Needs information from the user like configurations, links to their devices, or specific user instances.

The non-user-specific Alexa skills are fairly simple to create and for this guide I will be creating one using Node.js in an Amazon Lambda function (their version of a cloud run process) combined with an Amazon Skill.

  1. Set up a new Amazon Lambda function (Needs to be in N.Virginia in the top right of your dashboard screen)
  2. Set up a new Amazon Skill
  3. Grab the Application Id from the newly created Amazon Skill and replace the part in the AlexaSkill.js file relating to the ApplicationId.
  4. Then upload the zip file to your Amazon Lambda function.
  5. Then define the Voice Interface using the two files in the speechAssets folder. (IntentSchema.json and SampleUtterances.txt)
  6. In your lambda function, go to the Event Sources tab, and add the “Alexa Skills Kit” Event source.
  7. Then copy your Amazon Lambda’s ARN (Amazon Resource Name) and past that into the Endpoint textbox in the Configurations Tab of your Alexa Skill – something like arn:aws:lambda-us-east-2:9081209381:function:hello-world

The user-specific Alexa skills require an endpoint that allows the user to login to your own authentication service. This in turn will require a web app and a database of stored information per user. Stay tuned for more information on how to make that happen.

At the end of my investigation of the non-user-specific Alexa skill, I created and deployed a Wind Reporting service to Alexa that allows me to use the Echo to find out what the current Wind conditions are in my city before I take my drone out for flights.

linea_2ec(0)_512

Check out the Hackster.io post for the Alexa Wind Reporting Service: https://www.hackster.io/anthony-ngu/alexa-wind-reporting-service-7aada2

Publishing an Alexa Skill

Some things to keep in mind if you are making your own Alexa Skill for global publishing.

  • You will need icons for your Alexa Skill
  • You will need to process the generic Help intent
  • You will need to process the generic Stop / Cancel intents

 

Creating the Smart Home Universal Remote

It’s been a while since I last made a smart home device, not because my home is fully automated or because there wasn’t a need for another device, but because I still live in a rented unit and didn’t want to to spend the time making and setting up custom devices that would need to be torn down in the future.

Well the other day I realized that I could build another home automation device without a long-term stationary placement requirement! Not too long ago I built voice integration into my smart home system using the Amazon Echo (check out the articles here). While this worked well for moments without ambient noise, it failed to work well during parties, while watching movies, or while listening to music on my sound system. Obviously I needed another way to interact with these smart home devices and the current method of pulling out a phone or tablet, unlocking it, then switching between apps just didn’t appeal to me. What I really wanted was a universal remote that could also talk to my smart home devices.

So I started designing and planning out the features that I would want in my smart home controller and it had to be wireless charged (because replacing batteries or being tethered to a wall is archaic). Here’s the requirements I came up with:

Essentially, the goal is to get it all placed inside of an enclosure like this:

universal-button

Here’s a video of the very early prototype’s functionality:

As well as a more in depth Hackster post:

https://www.hackster.io/anthony-ngu/universal-smart-home-remote-wirelessly-powered-896f3c

Developing Mobile Apps with HTML/CSS/JS

I’ve always been interested in mobile application development and I learned Objective-C for iOS app development. However, as my web application development experience increased, I started to question the scalability of code for different mobile platforms (iOS, Android, and Windows). Developing separate applications for each platform with similar functionality but utilizing each platform’s separate SDKs and languages didn’t seem like the best method.

Because of this, I began to lose interest in native mobile development (Especially with the regular updates to the OS and slight changes that meant that I would have to keep updating the code for each new version) and I put off mobile development for web app development which could be scaled to work for mobile browsers using a blend of frameworks like BootstrapFoundation, AngularJS, and Node.js. Of course, making a mobile web app had some drawbacks. It loaded slower and would always need a connection to the internet in order to fetch the pages.

ionicview

That changed recently when I was re-introduced to Cordova and PhoneGap as well as the Ionic Framework. I had heard about Cordova when I was working with Objective-C but after a quick initial investigation, decided that it wasn’t for me due to the limited functionality in the earlier days. After a deeper dive this time, I’ve begun to see it as a platform that I can develop for.

It still uses HTML, CSS, and Javascript, but it’s a specialized project developed solely for mobile applications which allows for native app deployment rather than mobile website creation and can call the actual mobile platform’s SDK. It’s different than a webapp port, but it allows the web developer to use the same language (re-use big portions of code) and create hybrid applications for each platform that look like native applications and render through cached files.

Here is a great article about the history of Cordova & PhoneGap and the UI framework for them known as Ionic and here is an article about what the Ionic Framework is.

The biggest issue I reached early on was the need to talk to APIs and the CORS (Cross-Origin Resource Sharing) issue. Because some APIs don’t expect to be called client-side (which is what you code for in a Hybrid app), this causes some development pains. Rendering in the browser allows for fast iterative development and better debugging without having to rebuild each time, but will regularly throw the CORS issue in your face, while deploying to the phone will work perfectly fine but limit your debugging capabilities. To circumvent this, you need to open a new browser window with web security turned off (which is dangerous, so only use it for your application). More info on how to do this can be found here: http://superuser.com/questions/593726/is-it-possible-to-run-chrome-with-and-without-web-security-at-the-same-time

After that, I was in the free and clear and I’m starting more mobile development again.

Starting Development with Amazon Echo

Here’s a simple guide on how to create a Node.js app hosted in Azure that will handle your Amazon Echo‘s API calls.

amazon-echo

  1. You will want to download and install Node.js if you haven’t already.
  2. Download the code from the repository here.
  3. Create an Azure account if you haven’t already and create a new web app.
  4. Using FTP, Git, or whichever method you would like, get the code into the location for your new azure web app.
  5. Join the Amazon Developer program for the Echo and create a new Echo app. (Note: In order to use this while in development on your Echo, the account needs to be the same one that the Echo is linked to)
  6. In your App information tab:
    1. Fill out your “App Name”. This will act as your official app name.
    2. Fill out your “Spoken Name”. You will want this to be short and simple to say in order to give it the easiest time to recognize.
    3. Give your “App Version” which will need to match the info you hand back through the API.
    4. Give your “App Endpoint” which will be your Azure webapp’s URL + the api endpoint. (Example: “https://echotest.azurewebsites.net/api/echo”)
  7. In your Interaction Model:
    1. Fill out your “Intent Schema”. The intent is the name of the function, slots are parameters, and the type when “literal” will give you back the speech-to-text recognized word. More info on this here.
    2. Fill out your “Spoken Utterances”. They should be tab separated between the intent and the sample phrases. Something interesting to note is that they suggest that you provide a sample for every number of literal device phrases from min to max. (In my case from 1-3 words, thus the repetitions.) It also does not like it when you have multiple of the same literals anywhere in the file.. More info on this here.
  8. After this, set your app to be ready for testing and you are on your way!
  9. Call Alexa with your Spoken Name by saying “Alexa, open {YourSpokenAppNameHere}”
  10. Now you can say the commands that you’ve designated in both your Nodejs web app and your Amazon app declarations for your response!

If you want to make it your own, you will need to modify the Node.js back-end to respond according to the requests that you allow while also altering your intent schema and spoken utterances.

 

Amazon Echo: Should I buy?

amazon_echoThe Amazon Echo is a home automation maker’s dream. It provides an easy way to use voice recognition to interact with your devices, but there are a couple things you should know about developing for it before you buy one and start.

  1. Despite it being called an Echo “App”, your development will take place in a web-service hosted in the cloud that can answer it’s calls. What the Echo will do is translate what it hears into text and then hand it off to your service by calling your API with a package that contains the information.
  2. Creating an “app” with Amazon for the Echo requires you to fill out an “Interaction Model” which consists of an “intent schema” and “sample utterances” as well as program your web-service.
    • The “intent schema” is pretty straightforward and you basically create a JSON array of “intent” which contain a name and “slots” which are used like parameters and you must define the type.
    • The “sample utterances” are a list of the “intent name” and potential sample phrases.

Making it talk to a web service hosted in Azure using Node.js turned out to be fairly trivial and I was able to get a basic implementation hooked into the OpenSmartHub that I have been developing in less than a couple hours. I even created a sample in a github repository for those who want simple instructions and an easy place to start.

It really is amazing to see it come together and interact with your voice commands in a custom scenario that you have developed, but still has a long way to go in order to improve it’s voice recognition. It works really well with the pre-programmed functions, but there aren’t that many that I find particularly useful in an every day scenario and it doesn’t do well with brands or non-dictionary words. For example, it recognizes “Pandora” because it’s a vital part of their pre-programmed functionality, but it doesn’t recognize “Yamaha” or “Wemo” well.

Another thing that I’ve noticed is that it can sometimes mix up the singular and plural versions of words when converting text-to-speech. (For example, mine would sometimes hear “lights” when I say “light”)

Overall, I think it’s going to only improve from here and I think it’s worthwhile to invest into in order to integrate voice-recognition and voice commands into your homemade projects!

Node-OpenZWave on Raspberry Pi 2

As a continuation of my Open Smart Hub project I have been interested in adding Z-Wave and Zigbee devices to my supported devices and recently decided to swing for Z-Wave devices first. I bought a Z-Wave Z-Stick Series 2 USB Dongle from Aeon Labs and a simple Z-Wave Door Sensor in order to create the basic mesh network with just two devices.
z-wave-usb-stick

Since the Open Smart Hub is based in NodeJS, it only made sense to search for a Node port of the OpenZwave library. I stumbled upon Jonathan Perkin’s port of it to NodeJS (https://github.com/jperkin/node-openzwave)

Unfortunately, it does not work on Windows, and it seems to be having issues with the latest version of NodeJS… But luckily (or coincidentally) the Open Smart Hub runs on a Raspberry Pi 2 running Raspbian and NodeJS v0.10.28.

After the initial setup of my RPi2 with NodeJS, I got to work getting the node-openzwave module on my RPi2. I was seeing build errors when it was trying to install the module, but found a couple of blogs with information that in order to get it to work I might have to install a couple more tools.

sudo apt-get install build-essential make subversion
sudo apt-get install subversion libudev-dev make build-essential git-core python2.7 pkg-config libssl-dev

After that, it worked and I could call “npm install openzwave” and have it install properly.

Note: If you are interested in using it on Mac OSX, you will need to install the drivers for it. Read more about that process in a previous blog post.

Running NodeJS on Raspberry Pi

Getting NodeJS on a Raspberry Pi running the latest version of Raspbian isn’t as dead simple as it is on PCs, Mac OS X, or Linux. Only special versions of NodeJS were pre-built for Linux on ARM for Pi and a list of those versions doesn’t seem to be listed in an easy format.

In my case, I wanted to download NodeJS version 0.10.29 and the next closest NodeJS version for ARM was v0.10.28 or v0.10.30. You can search through each directory in the NodeJS distribution directory, but here is a small list of available Linux ARM versions when I last looked:

  • http://nodejs.org/dist/v0.10.28/node-v0.10.28-linux-arm-pi.tar.gz
  • http://nodejs.org/dist/v0.10.3/node-v0.10.3-linux-arm-pi.tar.gz
  • v0.11.12 through v0.11.0 all have Linux ARM versions for pi as well:
    • http://nodejs.org/dist/v0.11.12/node-v0.11.12-linux-arm-pi.tar.gz

In order to download them onto your Raspberry Pi you will need to type these commands into the shell, replacing the version for whichever you want:

wget http://nodejs.org/dist/v0.10.28/node-v0.10.28-linux-arm-pi.tar.gz
tar -xvzf node-v0.10.28-linux-arm-pi.tar.gz
node-v0.10.28-linux-arm-pi/bin/node --version

Now that you have it downloaded on your machine, you will want to add it to your path in order to allow it to be used from anywhere. So you will want to alter (or create if you don’t have one yet) a .bash_profile file in your root:

sudo nano ~/.bash_profile

then add these lines to the file:

Now after you reboot, you should see that you are able to use NodeJS regularly.

sudo reboot