Torching Igloos

Make your computer respond similar to “Jarvis” in the Iron Man movies *UPDATE 11/06/2012*

Posted by sneakily1 November 6, 2012, under Articles, Cool Technology, PC Tricks | 11 Comments

Iron Man

Make your computer respond similar to "Jarvis" in the Iron Man movies

(originally published Dec. 25, 2010)

 

**UPDATE 11/06/1012**

Added Youtube embed for tutorial that takes your Jarvis to a whole new level using some slightly more advanced techniques.

**UPDATED 12/30/2010**
**Added "Trash It" (delete a file) and "Empty Recycle Bin" macros**

 

Let me start off by saying Merry Christmas to all of my AMAZING Torching Igloos viewers! Your support means the world to me, and seeing as how this is our first Christmas together, I wanted to do something special. So my gift to you is a sweet tutorial on how to make your Windows 7 computer more human-like so you won't feel so lonely during the holidays. This is also a great way to show off for your friends... tell them for Christmas you got a talking computer that does whatever you want it to do. You'll be amazed how simple this is, requiring just a few free downloads, some MINOR scripting (seriously, my 5 year-old nephew can do it), and your imagination.

 

Before we get started, I have to confess... I originally did this a few years ago when I owned a Macbook. Yes, I know... Blasphemy! While Apple made the process relatively easy to do (even though finding documentation wasn't so easy), once I purchased my PC I found that this task was a little more difficult to address. When I say it was a little more difficult, I don't mean the actual process... trying to find information on how to make it happen was the difficult part. Fortunately for you, I've done the research and testing, so now you can partake with ease. Now on to the good part.

 

Required Materials:

 

* Windows 7 or Vista based computer (32 & 64-bit both work). I can't promise these steps are exactly the same for Vista (who uses Vista anyway?) so you're on your own if you're stuck with that crappy OS.* At least 2 gigs of memory. I believe it will work with 1 gig, but the more the merrier... and trust me, you want to have lots of memory for this feature.* At least a dual-core processor. Again, might work with less... but it's going to be brutal.

* A microphone. If you have a laptop with one built in, that's perfect... if you're on a desktop, something with good sensitivity would be best. A headset is fine I guess, but you're not going to be very cool trying to give your computer commands from across the room unless you have a wireless headset.

* Windows Speech Recognition (it's built in... I'll tell ya how to enable it)

* Windows Speech Recognition Macros (Now hosted here on Torching Igloos just in case Microshaft decides to kill it.)

* An imagination (this is the difficult part, if you don't think outside the box... it's not going to be as impressive).

* Speakers (yes, I know this seems obvious... but you never know).

 

If you've been keeping up with Torching Igloos, you'll remember I featured Windows Speech Recognition Macros in our article "More Awesome Free Programs!" we're going to use this program, and the speech recognition feature already built into Windows 7 (btw this should also work for Vista). So go ahead and download Windows Speech Recognition Macros. Before you install the Macros, make sure your microphone is attached and functioning properly (if you don't know how to do that, you're probably on the wrong site)... then go ahead and enable Windows Speech Recognition in your Control Panel (Start Menu/Control Panel/Ease Of Access/Speech Recognition/Start Speech Recognition). While you're at it, it's a good idea to take the speech tutorial to familiarize yourself with some common voice commands. You should also go ahead and visit the section that says "Train your computer to better understand you." Neither is required, but going through the steps provided by these tutorials will help make the process of bossing your computer around a lot more efficient. An additional bonus you'll get after training your computer to better recognize your voice... less problems with it responding to ambient noises and other voices in the room. So go ahead and take the extra 10 minutes to learn these processes.

 

You now have Speech Recognition working, and if you did the tutorials like I suggested... you're probably trying out all kinds of cool voice commands to make your computer function without even touching your mouse/keyboard. Surprisingly, yes... it's THIS easy! This is just the beginning of the fun though. Using Windows Speech Recognition Macros you can automate almost any function on your computer. So let's go ahead and install Windows Speech Recognition Macros. Just double-click it from where you saved the download (for fun, see if you can do it without touching your mouse/keyboard using only your voice). Then follow the proper prompts. Once you have it installed, make sure you have it create a desktop shortcut. Once this shortcut is created, I highly suggest copying it into your startup folder. To access your startup folder quickly, you can just pull up your Start Menu, click "All Programs, " scroll down to "Startup," right-click and choose "open." Then copy your Windows Speech Recognition Macros shortcut to this location. By doing so, you allow the program to automatically start whenever you boot your computer.

 

So what exactly is this Windows Speech Recognition Macros thing you've just installed? To put it in simple terms, it's a cool little program that allows you to create custom automations for windows that execute whenever you say a trigger word or phrase. This is where you're going to want to put your imagination to use. This is also where you "Teach" your computer how to talk back to you. I'm going to show you a couple little scripts you can make to get you started.

Let's start with something simple. Think of a name for your computer... for this example I'm going to name my computer Jarvis (it seems only fitting). First, make sure your Speech Recognition is enabled, and Windows Speech Recognition Macros is running. In your task manager (the bottom right-hand corner of your task bar) double-click the Windows Speech Recognition Macros icon and when the window pops up asking "What should the macro do?" click "Advanced." For those of you familiar with creating websites, this should be a welcome site. WSRM uses XML for commands... which makes creating custom commands relatively simple. For those of you who aren't aware of XML, it's still easy. Here's a quick synopsis of what you're looking at:

 

<speechMacros> - This is the starting Tag, it just tells the computer that this XML file uses the Speech Macros to execute.

 

<command> - This is the command to be executed. Each command is basically one action... in most cases you will only need one, but for some automations you'll want to include more than one. You'll see an example of this later.

 

<listenFor></listenFor> - This tells the computer what word or phrase to listen for in order to do an action. Notice that there is a closing tag (</listenFor>), this tells the XML that this spot in the code is where to stop doing whatever function... in this case stop listening for a word or phrase once the computer hears it.

 

</command> - The closing tag for the "Command" tag... again, in XML every tag will have an opening/closing tag and what happens between the two is how it knows what to execute. In this case it knows that the command is finished once the word/phrase has been detected.

 

</speechMacros> - By now I think you can figure out why this is here.

 

Since I'm going to name my computer Jarvis, I'm going to tell my computer to listen for it's name by changing the <listenFor> tags to say the following:

 

<listenFor>Jarvis</listenFor>

 

The entire macro looks like this now:

 

<speechMacros>

<command>

<listenFor>Jarvis</listenFor>

</command>

</speechMacros>

So my computer is now listening for it's name, Jarvis... but nothing is going to happen. We have to give it a way to interact, so we're going to add the <speak></speak> tags which allow for the computer to speak whatever text we insert between the tags back to us. I want my computer to have a bad attitude, so I'm going to add the following:

 

<speechMacros>

<command>

<listenFor>Jarvis</listenFor>

<speak>You're not Tony Stark! ALERT! ALERT!</speak>

</command>

</speechMacros>

Go ahead and hit "Save" then "Create" and then "OK." You might want to make sure you know where WSRM is saving your macros before you hit "Create" though... I just save mine in My Documents under a folder called "Speech Macros." Once you save these you can easily edit them by just double-clicking on the macro file.

 

You've now created your first speech macro. Now let's test it out! If you don't have your computer already listening for voice commands, just say "Start Listening." Once your computer has acknowledged that it's in listening mode, Now say "Jarvis." If you've done everything correctly, your computer will say "You're not Tony Stark! ALERT! ALERT!" in a female voice. If you're not a fan of the female voice, back in your control panel where you setup your speech recognition, on the left hand side there's an option called "Text to Speech" click on this and under the "Voice Selection" tab there's a couple you can choose from. Hopefully more people will start using speech recognition and force Microsoft to release cool voices. Wouldn't it be awesome if you could have Jarvis' actual voice or Darth Vader... or something cool like that? Go ahead and play around a bit with the <listenFor> and <speak> tags and create some macros that give the sense of interaction with your computer. On my old system... since I worked in a bar full of foul-mouthed rockstars, I had various four letter words programmed in so when someone would cuss, my computer would say "You shut your mouth when you're talking to me!" Just a little example of some of the fun you can have.

 

Making your computer respond with words is fun, but it can get old quick. So let's have your computer do some other functions using voice commands and macros. One of my favorites is a weather script I modified (it was originally on a Windows 7 forum so I don't take credit for writing this... only for modifying it for my own needs). This script is a little more in depth, but look through it a bit and see if you can figure out what it's going to do. Notice it's a two-part macro. (in case the pasted script doesn't work properly due to spacing/cut-n-paste factors, you can just download this macro here and extract it using winzip or some other utility.)

 

<?xml version="1.0" encoding="UTF-16"?>

<speechMacros>

<command>

<listenFor>Pull up the weather in [CityName]</listenFor>

<run command="http://www.weather.com/weather/local/{[CityName.zipCode]}"/>

</command>

<command>

<listenFor>Is it cold in [CityName]?</listenFor>

<speak>Let me check... just a moment.</speak>

<script language="JScript">

<![CDATA[

<var xml_doc = new ActiveXObject("Microsoft.XMLDOM");

<xml_doc.async = false;

<xml_doc.load("http://www.rssweather.com/zipcode/{[zipCode]}/rss.php");

<var titles = xml_doc.getElementsByTagName("title");

<var descriptions = xml_doc.getElementsByTagName("description");

<Application.Speak(titles.item(2).text + " in {[*CityName]}");

<Application.Speak(descriptions.item(2).text);]]>

</script>

</command>

<listenForList name="CityName" propname="zipCode">

<item propval="65201">Columbia</item>

<item propval="37201">Nashville</item>

</listenForList>

</speechMacros>

This macro does 3 things:

* Listens for the phrase "Pull Up The Weather In [City Name]" thus, starting up your default browser and sending it to weather.com with the city you named.

* Listens for the phrase "Is it cold in [City Name]?" thus, reading you the weather forecast for the mentioned city.

* Assigns the city name specified to the proper zip code so the computer can retrieve weather information for the designated city.

 

This macro also uses javascript to decipher some of the functions... but that's not important right now. I'm not a huge fan of javascript myself, and I certainly don't expect you to use it for your everyday macros, but in this macro it serves it's purpose. Now if you've actually looked at the code, you'll notice it has some serious limitations. First of all, it's only designed to choose from one of 2 cities... Columbia, Missouri (where I currently live... yes, feel sorry for me) and Nashville, Tennessee (where I used to live and should have stayed... AWESOME town!). So if you want this macro to choose from your own choice of cities, you need to change the city names and zip codes in the <item> tags. Someone who knows more about actual Javascript can probably come up with some better code that listens for any city and converts it into the proper zip... but obviously the author and my own skills are limited, so we'll just work with what's available. Besides, if you don't tell people... nobody will know that it's just simple scripting. It's not how the magic trick happens... it's how amazing the illusion seems to the target audience.

 

Having the weather forecast read to you is pretty cool, I've experimented a bit with some other text-to-speech automation scripts using RSS feeds (News, Sports, Etc) and I've found that while it actually works... most RSS feeds toss a lot of code into the descriptors, so sometimes it can sound pretty messy. You'll have to play around a bit and see what works best for you. If you come across anything cool along these lines, please let me know. I'm always looking to improve my speech macros. I'm also going to spend some time cleaning up our RSS feeds here at Torching Igloos and I'll create a macro for those so your computer can read you the Torching Igloos news.

 

While I was researching the various speech recognition functions of Windows 7, I was appalled to discover that some of the most common Windows tasks aren't supported (or are difficult to find information on). Some examples: Restart, Shutdown, Volume Up/Down, Empty Recycle Bin. You would think that as basic as these tasks pose to be, Microsoft would have included them in Speech Recognition. Well, if they did I certainly couldn't figure it out (hell, Einstein couldn't tie his shoes so it is completely possible that I'm just missing something obvious). So, here's a few macros to make some of these tasks simple. Unfortunately a few of them do require some outside assistance in order to function properly.

 

Restart Computer: (You say "Nuke It" and the computer acknowledges and restarts your system.)

 

<speechMacros>

<command>

<listenFor>Nuke it</listenFor>

<speak>Restarting Windows</speak>

<run command="C:\Windows\System32\shutdown.exe" params="-r -t 00"/>

</command>

</speechMacros>

Shutdown Computer: (You say "Shut It Down" and the computer acknowledges and turns the system off)

 

<speechMacros>

<command>

<listenFor>Shut it down</listenFor>

<speak>End Of Line</speak>

<run command="C:\Windows\System32\shutdown.exe" params="-s -t 00"/>

</command>

</speechMacros>

** Trash It (delete file): **

 

<speechMacros>

<command>

<listenFor>Trash It</listenFor>

<sendKeys>{DELETE}</sendKeys>

<speak> File Deleted</speak>

</command>

</speechMacros>

System Volume Macros & Recycle Bin:

 

These require a little program called nircmd that executes windows command-line functions in the background. Unzip to c:\nir\ or change the command line in the macros to reflect your nircmd.exe location.

 

Volume Up:

 

<speechMacros>

<command>

<listenFor>volume up</listenFor>

<run command="C:\nir\nircmd.exe" params="changesysvolume +5000"/>

<speak>Volume Up</speak>

</command>

</speechMacros>

Volume Down:

 

<speechMacros>

<command>

<listenFor>volume down</listenFor>

<run command="C:\nir\nircmd.exe" params="changesysvolume -5000"/>

<speak>Volume Down</speak>

</command>

</speechMacros>

Volume Mid:

 

<speechMacros>

<command>

<listenFor>volume mid</listenFor>

<run command="C:\nir\nircmd.exe" params="setsysvolume 30000"/>

<speak>Volume at medium listening level</speak>

</command>

</speechMacros>

Volume Low:

 

<speechMacros>

<command>

<listenFor>volume low</listenFor>

<run command="C:\nir\nircmd.exe" params="setsysvolume 10000"/>

<speak>Volume at low listening level</speak>

</command>

</speechMacros>

Mute:

 

<speechMacros>

<command>

<listenFor>mute</listenFor>

<run command="C:\nir\nircmd.exe" params="mutesysvolume 1"/>

<speak>Volume muted</speak>

</command>

</speechMacros>

Unmute:

 

<speechMacros>

<command>

<listenFor>unmute</listenFor>

<run command="C:\nir\nircmd.exe" params="mutesysvolume 0"/>

<speak>Mute Off</speak>

</command>

</speechMacros>

** Empty Recycle Bin: **

 

<speechMacros>

<command>

<listenFor>Empty Recycle Bin</listenFor>

<run command="C:\nir\nircmd.exe" params="emptybin"/>

<speak>Your junk has been recycled</speak>

</command>

</speechMacros>

Some other cool things to note while we're playing around with speech recognition, "Mouse grid" which brings up a grid for mouse movement, and a cool Firefox add-on called Firesay which allows you to control many of Firefox's functions using voice commands. With all of these new tools, and some creativity you can have your computer saying/doing all kinds of cool things. Try setting up play lists to suit your moods so when you come home from work, you can just tell your computer what kind of mood your in and it will automatically load up your favorite music player with the songs you want to hear. Invite over your girlfriend and impress her with music cued to certain words you say during a romantic dinner you've prepared for her (btw I'm single ladies). Setup your media center to start movies just by saying "I want to watch (insert movie name)" the possibilities are endless, and the ability is now yours. Now go impress somebody with your new found wizardry!

 

**Update 11/06/2012**

I found a tutorial on Youtube where the person gives an excellent tutorial on how to take your "Jarvis" to an even higher level of interactivity. Using my method outlined above, and some of the tricks this kid uses... you should be able to create some really fun interactions with your computer now!

 


Enhanced by Zemanta

Currently have 11 Comments

  1. Damnit. all your download links expired on 5th of february. and im here now :/

    however i got the macros but you only put one dead download link to the second something.

    Is there another place i can get that?

    its the weather link thingy.

    • I didn't even think about that, sorry! I'll change it right now and browse through the site for any other urls that might have expired due to torching.me dying. Thanks for pointing this out!

  2. Hello your tutorial is really good but i have problem with my speech recognition program that is most of the time it can't understand my voice i have tried a lot of time but out of 100 may be 10 time it understood my voice saying Jarvis otherwise it kept on saying what is this please setup a video tutorial of method the program can recognize my voice and a video from the beginning of this tutorial like from the start of setting up the speech recognition program and then come to this topic thank you

    • sneakily1
      sneakily1 says:

      The Windows Speech Recognition engine is rather dated and there isn't much a person can do to fix the problem of it not understanding what you say. Honestly, Microsoft has completely dropped the ball on speech recognition if you ask me. All I can recommend is doing the tutorial under speech recognition in your windows7 control panel so it can try and learn your voice better. You might also try changing the language settings in windows to your native language and giving it a shot that way if English is a second language for you. I'm not exactly sure that this would work... but from what I've read here and there, it seems that if you choose it to recognize english and you have a thick accent (even Boston people complain about it)... it fails miserably. Once again, a huge failure on the part of Microsoft.
      Personally, I've become a big fan of Android for it's speech recognition and text-to-speech abilities... far superior to any half-assed, unsupported feature Microsoft tossed into Vista and Windows 7.

  3. Your tutorial is great! I will have to set this up on my windows. I use my macbook for everything so do you have a link or a page dealing with programming it for OsX? Also, how might you add multiple triggers to one response instead of making it over and over again, this is what I'm thinking of. Trash it; Delete it leading to the response. And one more is there a way to make it branch out to add a AI feel to it and have conversations. So something simple, you say "empty recycle bin" response "are you sure" you say "yes" then the recycle bin is emptied. The main thing I am looking for is how to create a personalized vocal response system for the Mac (The incorporated's no fun and sketchy), I've looked everywhere and this is the closest thing.

    • Ben, thanks for the kind words! For the Mac, here's a tutorial to get you started, it's been about 4 years since I had my MacBook, so I don't remember exactly the process of making it speak back... but I remember it was something to the effect of just creating a file and typing in the phrase you wanted your Mac to say back to you, then name the file the command you want it to recognize in order to execute. An example of this would be creating a file called "Backwards" (the trigger word) and then the text saying "Why would you want me to walk backwards? (the response).
      For multiple triggers in Windows Speech Recognition, watch the video at the bottom of the post, the guy giving that tutorial explains how to achieve that a little better than my tutorial (3 trigger words performing one task). If you're wanting to string commands together, you'd probably have to do some more complex scripting using if/then statements. I'm not sure exactly how you would go about doing that off the top of my head, but I'm sure it's possible. I'll see if I can find some more information on such a thing and maybe post it as another update. Let me know if that helped answer your question.

  4. Good to see somebody else in the world interested in speech activated scripting/macroing. I have been doing speech activated scripting for many years, using NaturallySpeaking with Dragonfly (uses Python format macros, and it's a bear to install). Moving to Windows 8, and I might switch to Windows Speech Recognition Macros.

    If you think that's difficult... Can you speak "gee" (or whatever) to have the mouse button held down. Then move the mouse as clicking and dragging. Then speak "ex" (or whatever) to release the mouse button? In other words... Clicking and dragging by voice.

    Also. Does it work in a full screen game?

    I guess you are familiar with the Microsoft site for WSRM. If you know of any other active discussion forums on the subject, please let me know.

    Thanks.

    • As for the clicking and dragging, I've never thought to test that out, I'll do some research and see if I can find something. As for gaming, not so much, since games use their own control schemes and windows hasn't done anything to push developers into integrating such a thing with DirectX or something. I'm hoping that Android starts jumping on that bandwagon a bit with the voice recognition engine and starts making games you can yell at. :)

      • I found this, which gives a bit of an idea how to do the click and drag. Not sure how you'd put it into your macros per-se but it should give you an idea where to start http://torching.me/1yg (scroll down to 3rd forum entry)
        I also found this entry http://torching.me/1yh, the part about gmote is interesting because perhaps there's a way to run command-line params with it to integrate it into WSRM. I don't know... but I'll give it a try and see what kind of results it could come up with. It'd be cool to create maybe a series of gestures and then have WSRM execute it using voice. Like I said, not sure if it's possible because I haven't tried it yet.

  5. sneakily1
    sneakily1 says:

    looks like you put a lot of work into those macros. Nice work

You must be logged in to post a comment.