Simple and powerful web service access with YQL

Posted by fulvio at Oct 09, 2012 02:05 PM |

During Eric Brehault's excellent Plomino training at this year's Plone Conference I became acquainted with a new, exciting bag of tricks:  the Yahoo! Query Language.

Exhibit A: The Query

select * from weather.forecast where woeid in (select woeid from geo.places where text="arnhem")

Looks like a SQL statement, you say?  You would be forgiven, but only if you try this:

YQL Request

Notice that the query string is essentially the previous "SQL statement", after URL encoding.

The response to the above HTTP request is JSON data, encoding the weather forecast for Arnhem, Netherlands.  Are the little gears in your brain turning yet?

Exhibit B: The Console

While the link above resulted in raw data, the following link takes you to the YQL console:


Notice the following:

  • The large text box at the top is pre-populated with the previous query
  • Directly under it, click the Test button, and try switching between XML and JSON, as well as between the Formatted or the Tree representations.
  • The two right tabs allow you to experiment with the two individual data sources that are joined by the query.
  • Finally, in the text box along the bottom you can find the REST Query I linked above, which returns the raw data.

The Proof

The little weather icon to the left was generated with a small snippet of JQuery utilizing the same query URL from above.  Note how no javascript API is loaded from remote sources, and we are combining two different web service data sources in a single AJAX call.  Safe and fast.


Piecing it together

Go back to the YQL console, and drill down into the list of Data Tables on the right, until you find weather.forecast.  The large text box at the top will be populated with a sample query:

select * from weather.forecast where woeid=2502265

Here, woeid=2502265 represents Sunnyvale, CA.

Next, go back to the Data Tables list, and click on geo.places.  This time, the sample query is:

select * from geo.places where text="sfo"

Copy the query, and go back to weather.forecast.  Instead of woeid=...., let's use the in operator, and put a pair of brackets around the 2502265 value.  Finally, replace the 2502265 value with the query from geo.places:

select * from weather.forecast where woeid in (select woeid from geo.places where text="arnhem")

That's how easy the console makes it for us to discover how to piece together any web service query we can think of!

Finally, it's just a matter of pulling out the pieces of data from the JSON response with a little bit of JQuery.

Of course, by playing around with the console, or even reading the extensive YQL documentation, we can make the queries much more efficient and optimized, but this is a great start.

If you want to use data from such disparate sources as Zillow, Craigslist, Flickr, Pidgets,, Wordpress, Yelp, Facebook, Twitter, YouTube, Answers, and many others, I can't recommend YQL highly enough.

seth williams
seth williams says:
Sep 01, 2014 10:16 PM
Hey Fulvio, thanks for sharing these exciting tricks on accessing the web service with YQL.
Commenting has been disabled.