Google I/O 2011: Writing Web Apps in Go

Google I/O 2011: Writing Web Apps in Go

ROB PIKE: Hi everybody. Well, thanks for coming. We’ve got some important things
to talk about today. This session is called Writing
Web Apps in Go. My name is Rob Pike, I’m
presenting with Andrew Gerrand, and we’re both members
of the Go team at Google in Mountain View
and in Sydney. During this session, if you have
feedback you’d like to give, or other items you’d like
to talk about, there’s a link to the– the short link will give you
a site for feedback. And then, here’s a couple
of hash tags that you might want to use. But you could also use
the golang hash tag, we won’t get mad. So some of you might not know
what Go is exactly. So I thought we’d spend just
a couple of minutes talking about that. Go is a programming language
for the world of today. If you look at what happened
in the last 20 years or so, the computing environment has
changed a lot, and not all the languages have really
kept pace. So Go is designed inside Google
to be a fun, efficient and productive way to write
kind of software that we write at Google. Which mostly means things in the
manner of web servers and things like that. Most importantly of all
though, it’s an open sourced language. Everything about it is
done in the open. It is not a proprietary thing
in any important way. It launched in November 2009 and
it had the goals of being fun to work in, efficient,
scalable, and very, very productive. And as I said, we designed it
for web servers, but it sort of works really well for
just about anything. Which is a bit of a
surprise to us. And so we now prefer to just
call it a good, general purpose language. And there are contributors now
from all around the world working on it. So today, I’m going to talk a
little bit what’s go on since the last talk at IO
over a year ago. We’re going to show you a web
server that’s written in Go and how to deploy Go
in production. You might know a little bit
about what that’s about today. We assume you know a little
bit about how to write web servers, maybe a little
bit about Go, but it’s not totally important. You’ll get the gist of it even
if you’re completely new to the language. We’ve got some big
announcements, some of which have already leaked, which
is fine with us. And there’s some
giveaways too. Last year we giveaway t-shirts,
maybe we’ll give something else away today. So be sure to stick around to
the end when all the fun really comes together. So what’s happened
in the past year? It’s been a really
big year for Go. Tons of stuff happened. Perhaps the biggest news of
all has been that it’s now supported on Windows, entirely
through the efforts of an open source community. We did none of this work
internally on the team at Google, which is very, very
satisfying for us. There’s also a couple of
external application– sorry, external compilers
that have been done. There’s a compiler in the works
called Ergo, and there’s also an interpreter with JIT
that runs on Windows and is now starting to support .net. It’s called Express Go. We’re really excited
about that. Our own internal port
for the ARM is now very solid and available. It works on all kinds of phones
and things like that. GCC4.6, that release
includes GCC Go as an official language. It’s included with things like
Ubuntu and stuff like that. There’s SWIG support and
on and on and on. For those of us who write code
for a living, it’s nice to be able to say that GDB is now well
supported or Go is now well supported by GDB. And here’s a sort
of silly trace. It’s kind of hard to show a
GDB slide that’s not just modem noise. But you can see– do you know what modems are? Are you guys too young? You guys will recognize things
like the run time knowing about Go routines and you can
talk about threads, as well as Go routines and channels. And it’s really quite nice. It’s all symbolic debugging
now in GDB. And that’s also getting
better all the time. In the past year, the language
has simplified. Unlike some other languages we
could name, we’d like to keep Go really clean and simple. And to that end, we’ve actually
removed some things from the language in order to
make the language better, which I think is a sign
of the kind of stuff we really care about. But the language actually gets
better through doing this. So non-blocking communication
operations went away because they were confusing
semantically. And it turns out the select
statement can provide all you need there anyway. We got rid of the closed
built-in because people tended to misuse it. It was very subtle to use
well and veru racy. We clarified the type sizes
for floats and complexes, which simplified some– made the math libraries
much nicer to use. Slices got simpler. The dot dot dot thing, which
is how you do variadics actually got a lot nicer. And we also put in an append
built-in, which clarified a lot of code. Comps and literals got
easier to type. But basically, the thing just
got sort of more regular and I think a lot cleaner. And it continues to develop. As I mentioned many times, it’s
entirely open source. I mentioned the Windows port. We’ve had over a thousand
contributions from more than 130 people around the world. We won a best of open source
award from InfoWorld, which was really nice. And I think what really pleased
me about this award was besides getting the award,
the citation really seems to understand what it is that
Go is trying to do. So it was very satisfying. Now. When we first launched Go in
November of 2009, we called it a systems language. And that seemed to
confuse people. People seemed to think that
means it’s for writing really low-level, grungy code,
operating systems, maybe file systems, things like that. And I think Go is an interesting
language to do those in, but we had
a somewhat higher level, grander idea. It really was for the kind of
software we write at Google. But as I said before, it was a
bit of a surprise to us that it turned out to be a great
general purpose language. And people have been using it
to do a lot of scripting, a lot of sort of administrative
stuff, and various things that we really didn’t have in mind. But it’s turns out
to be good for. That said, it’s still really
good for writing systems code, like web servers such as
the one that Andrew will now show you. Andrew. [APPLAUSE] ANDREW GERRAND: So my name’s
Andrew and I’m going to show you the development of a
complete web application called Moustachio. And Moustachio is our bid into
this burgeoning market of photo and image sharing. Good way to get venture
capital I hear. The core functionality of this
program is the ability to upload an image or a photograph,
to draw a moustache on that photograph,
and then to share it with your friends. In this way, I hope to
demonstrate how Go shines at serving web pages in templated
HTML, handling file uploads, working with the file system,
and importantly, manipulating and decoding and encoding
images. And we’ll also show
interacting with OAuth-authenticated APIs. So this is the simple
hello web server. If you access this web server on
port 8080 you will see the string hello world. And so the way this works
is we import the string formatting in HTTP packages,
we define an HTTP handler. And what the handler does is
simply write the string hello world to the HTTP response
writer and that sends that string as the HTTP response. And then in our main function,
we have a handle func call, which registers that handler
to our web root. And then a listen and serve call
to actually set up and run the web server. And so the beginning of our
Moustachio program will simply handle a file upload and show
the image to the user. And this is the request and
response flow of that process. So initially, when the user
visits Moustachio, they issue a get request and the
upload handler will serve them a form. And the form will include a file
input field, and then the user will post an image back
to the upload handler. And the upload handler will
save the image to disk, redirect them to /view with the
image ID, and then when the user requests /view, the
view handler will serve them the image data proper. So here’s our simple
HTML form. I imagine it’s familiar
to most of you. It’s just a simple HTML form
with multi-platform data encoding type and a file
field named image. And so to create an upload
handler that displays that form, what we’ll do is import
the template package and as a global variable, create the
variable upload template, which is the result of
calling must pass file on that template. So in our new upload handler, we
can call the execute method on that template to actually
write the template out as the HTTP response. And then in our main function,
we registered that handler to the web root. And now we need to expand that
upload handler to also accommodate handling
the uploaded file. So our original code for the
upload handler is still there under this if statement, if
method is not equal to post. So if they’ve just done
a get request, we serve them the form. If not we’re handling a post
request. And so we pull the form file out of the HTTP
request, and then we open a temporary file using the
temp file function from the iUtil package. And then we copy the contents
of that form file upload to our temporary file, and then
redirect the user to /view with the name, the automatically
assigned name of that temporary file. And so one thing that is
important to notice here is that there’s a lot of error
handling in this slide. And in Go we consider error
handling to be very important because unhandled errors
lead to bugs. But it doesn’t have to
be this verbose. And so let’s look at a
way of just cleaning up the error handling. We can clean it up a bit by
creating an error handler function that wraps our HTTP
handler and returns a new HTTP handler that will catch errors
effectively, and display a nice HTML page with
an error message. And so that’s what this error
handler function does. It has a corresponding
error template. And in the same style as
the upload template. And then the important part here
is on this first line of the main function. We now wrap the upload function
with our error handler function. And then we modify our upload
function to use this new check function, which will panic if it
encounters an error, which will then be caught by
the error handler. And so you can see that compared
to a couple of slides ago, the error handling
is much simpler. Is just cause to check
parting in the error. And so now we need to define
a handler view to actually serve our image. And that will simply set the
content type to image so the browser will display
it as an image. And then call the serve file
helper to read the file from disk and write it to
the HTTP response. And then we need to add it with
handle func to our HTTP [? muxer. ?] And so I can demonstrate this
in action for you now. If I just running locally
on my machine. I’ll upload an image. And there it is being
served back to me. My little friend here. So now let’s talk about
moustaches. To draw our moustache, we’re
going to use the freetype-go library, which is the Go
implementation of the FreeType font rendering library. And it has a rasterization
package called raster that draws– gives us anti-aliased curve
drawing primitives. And to use this library we
install it with Goinstall, which is out part of the
Go distribution. It’s a tool for automatically
downloading and installing Go libraries. And we use goinstall
to install it. And once that’s done, we can
import the package from the same name that we Go installed
it from, which is nice and convenient. And so to actually draw our
moustache, we’ll use three Bezier curves. And we’ll allow the user to
control the droop factor of the moustache by moving some
of the control points. And so the function to actually
draw the moustache looks like this. This is the first half. It takes an image.image, which
is Go’s internal image format. Or part of the standard
library rather. It takes x and y coordinates for
the moustache, a size for the moustache, and the droop
factor for the moustache are very important. And at the top of this function
we’ve set up the rasterizer to paint onto our
image and set up a new rasterizer. And then this block of
essentially, just mathematics to define each of the control
points involved in drawing the moustache. It should be kind of familiar to
anybody who does any fixed programming I guess. And then on the next slide, we
actually draw the three curves from the left, around to the
right, and back down again. And at the bottom of the
function, we call rasterizer to actually draw to our image. And then we return the
resulting image. But there’s one little detail
about that function that I haven’t discussed. And that is that an image.image
value is an interface type in Go, which
means that it can represent any type of image. And that’s appropriate in this
case because a user can upload any [INAUDIBLE], kind of valid image. And that might be in a variety
of different color spaces. It might be RGBA or a greyscale,
or a fixed-palette GIF file and so on. But the raster library works
best when it’s used with an RGBA image. It’s much easier to set up. And so at the top of the
moustache function, I have this call to the
function RGBA. And it passes in the provided
image and returns an RGBA version of that image. So let’s take a look at how that
function actually works. The RGBA function first,
performs a type assertion. That’s this first line of code
in the function there. But what this assertion is doing
is asking the interface, is your underlying concrete
type an RGBA image? And if it is, then this means
that we can just return that concrete type, we don’t have to
do anymore processing, and that’s the fast path
through here. Otherwise, we create a new
RGBA image and use the draw.draw function to actually
draw the existing image to our new RGBA image. And then return it. So this is a very idiomatic
way of introspecting into interfaces. It becomes very handy in
a lot of situations. So now that we have our
moustache drawing function, there’s the matter of
integrating that function with the rest of our HTTP flow. So the first part of this
diagram is the same as in the first example. But then, instead of redirecting
to a view handler to simply view the image, we’re redirecting to an edit handler. And this edit handler will
display an HTML page, which includes some JavaScript and
that JavaScript will provide edit controls and allow you to
place the moustache on the image and adjust it. And with each time you adjust
the image, adjust the moustache and move it around,
the image tag that shows that image will be updated to include
a new query string with these x, y, s and d
parameters to signify which parts of the image
should be shown. Where the moustache on the
image should be drawn. And then the image handler
will actually draw the moustache and return
that image. And so we’re doing all of the
rendering of this moustache on the server side, but updating
it in real time. So the edit handler
is very simple. It’s much like the first vision
of our upload handler. We just set up an edit template
reading edit.HTML. And the one difference is we
pass in the ID from our queries using the form value
method on request. So that the JavaScript can know which image
ID it’s referring to when it’s drawing the
image on the screen. And our image function
first opens the relevant file, image file. It decodes that image, then
it pulls the x, y, s and d parameters from the query
request. And then uses the moustache function to
draw on the image. And then it will serve the
resulting image as an encoded JPEG at the bottom
of that function. You see this is kind of a bit
repetitive and kind of unnecessarily verbose. We can sort of clean that up a
bit and make it a bit more general by just using
a helper closure. And we did find this closer
v, that takes a string and returns an integer. And that does the string
conversion from strings to ints. And this is necessary
because Go is aesthetically typed language. And so when we call moustache
and we use v of x, v of y, et cetera. So I’ll just demo that now. The fun part. Upload our image, again. Now we actually have the
edit controls in place. I can place the moustache. And you’ll see as I place the
moustache, as I move it around, here the image
query string changes. I can adjust the size, make it
big and bushy, or a little bit more sort of thin
and dignified. And also, adjust the droop
factor, the all-important. You can kind of go a little bit
extreme if you want, or all the way. I think it’s a happy occasion,
so I”ll make it slightly upturned gentlemen. So now that we’ve drawn our
moustache, it’s time to move on to sharing this hilarious
image with all of our friends. So this is the latter part
of the HTTP flow. From our edit.HTML template, and
we’ll have a share link, which you might have seen when
I was demoing just then. And that share link will link
to the /share handler and passes in all the definition of
that image with moustache to the share handler. And the share handler creates
a redirect URL to Google’s OAuth service. And if you’re familiar with
OAuth service, this should make sense to you. And I hope it makes sense
to you regardless. And the share handler redirects
you to Google’s OAuth service, and then the user
will perform the little authentication and permission
granting dance with Google’s OAuth server, which usually
means logging in and clicking an accept button. And then the OAuth service
redirects the user back to our Moustachio App to /post, which
is our post handler. And it passes in an
authentication code and the image stayed. Which is those query parameters
from before. When the post handler receives
that data, it takes the authentication code and performs
its own interaction with the OAuth service,
exchanging that code for an authentication token. And then it uses that
authentication token to make its authenticated post
to Google Buzz. And then if that response
comes back OK, our post handler will draw a
congratulations, your image has been shared HTML
page and show the image back to the user. Fortunately, it’s a lot
less complicated in the actual code. And to share the image on
Google Buzz, we need to authenticate with
OAuth as I said. And to do that, we’ll use
the goauth2 library. And to install that again,
we can use goinstall. And again, we can import
it under that path. And to set up the OAuth
client, we need some configuration. And so we set a global variable
config, which is appointed to OAuth config. And we set the client ID and
client secret, which are obtained from the Google APIs
console or whoever your OAuth provider is. And we set some URLs to specify
authenticating with the Buzz service. And that we want to use Google’s
OAuth service. That’s what those three URLs
are. and the final and important URL is the redirect
URL, which is where OAuth should send the user back to
when the authentication exchange is completed. And that’s our post handler. So our share handler
is very simple. It just looks like this. We call the OAuth config URL
method on our config object. And passing in the raw query,
which is our image definition and then redirect the
user to that URL, to Google’s OAuth service. Once Google has redirected the
user back and they hit the post handler and the post
handler takes the authentication code
in the image. And at the top of this function,
we set up an OAuth transport with our
OAuth config. We take the code and call the
exchange method on that transport to authenticate that
transport with an access token, and then we call our post
photo function, which I haven’t described yet passing in
an HTTP client made by the OAuth transport and a
URL that we want to post to our Buzz stream. And finally, if all of that
succeeded, we use the– we execute the post template,
which will be our screen saying, congratulations, your
image has been shared. So the post photo function is
fairly straightforward. It just forms a JSON-encoded
request, which I’ve omitted from this slide because
everybody seems to know what a JSON blob looks like. And then, makes an HTTP post
to the Buzz API using the provided HTTP client,
which is our OAuth authenticated HTTP client. So we just call the post method
on client to the Buzz API URL and type application
JSON and our JSON-encoded request. If an error occurs, we return
that error code or a custom error code. Or we just return nil if
everything went fine. This is more than you’ll need
to do in the future to interact with Google APIs. A more general Go client for
Google APIs across the board is being developed. And it should simplify a lot
of this kind of code. So let’s demo the sharing. So I have my gopher image. And I want to share it. So I click share. I get sent to Google and I’m
asked, Moustachio’s requesting permission to manage
your Buzz activity. I allow the access. And then it says, your image
has been posted. And now if I go to my
Buzz stream, it should magically appear. And there it is. Ah, but the image is broken. And the reason why is because
if you look at the URL, I’m running on local host 8080 and
the Buzz service can’t actually access my local machine
running on a private network to grab that
thumbnail. But when we move it to
production, I hope that this problem will be alleviated. So to talk about going to
production, I invite Rob back to demonstrate that. Thanks. [APPLAUSE] ROB PIKE: Thanks Andrew. While I was sitting back there
and Andrew was talking, I noticed there were relatively
few laptops open, but a surprisingly large number
of tablets. I don’t know what
that’s about. So once we have our little
Moustachio program running locally, we obviously want
to deploy it for the world to enjoy. And of course, that requires
a platform for doing it. And so we’re delighted today
to announce that Go is supported on Google
App Engine. [APPLAUSE] ROB PIKE: Which means he’s
going to need his flight goggles, or he’s not going to be
able to be ready for this. OK. Now, I don’t know how
many of you are familiar with App Engine. It’s definitely a slightly
different way of building the app. But it’s fundamentally
the same ideas. It’s just some things are
different, some of the APIs change a little bit. So I’m going to talk in very
brief terms about the changes you need to make to the code
that Andrew showed in order to have it run on App Engine. One of the big differences is
that in the App Engine set up for Go, the main function is
actually run by App Engine. So the thing you write is not
a main, it’s something that main calls. So the first and obvious change
is that we have to make a different package. So we call it package
Moustachio. And then we have to import a
couple of the App Engine relevant data APIs. The one we’re going to be using
today is Data Store. You might have another
three or four depending on your program. And then we take the set up for
HTTP and move it out of our main function and
put it into an init. But that’s a very mechanical
change. There’s really nothing
interesting going on there. It’s very simple. The big change happens because
on App Engine you can’t read and write files. And it’s sort of a fundamental
thing to the way App Engine works. A request might go to one
machine and then the next request of the same
clients goes through a different machine. If you write files, it’s going
to be in the wrong place. So we just don’t do that. Instead, there’s a distributed
thing called data store and we store the image we’re going to
play with inside App Engine’s data store to make that work. To do that, we need to define
the piece of data that we can store inside the data store. And we call that an image. And this isn’t a very
interesting one. It’s just a block
of data stored inside an image structure. We could in fact, use blob store
for this app, which just is a bag of bits. But we made an image because the
data store’s a little more interesting to work with
because it allows more experience if we want to add
providence information or other details to the image,
we could do that. Blob store would make
that harder. So now what we need to do is
figure out how to store the image inside the data store. And to do that, we need to
define a key that we can store the image under. And the easiest way to do that
is just to create a unique key from the data itself. And we can do that. Here’s a really simple
function. Actually, quite a lot
going on in here. I’m not going to go through
it in detail. But what this does is take
a string of bytes, it’s presumably the data from the
image, compute the SHA-1 hash of it, and then just take a
chunk out of that SHA-1 hash, hexify it, and use that
as a string that we can use as the key. It’s truncated 16 bytes
of hex here. Obviously, you could make it a
full SHA-1 string, but then you get a URL a foot long and
we probably don’t need that much protection. So that’s a parameter
in principle, you could make it longer. Now once we’ve done that,
we have to take the file management code that Andrew
showed you and convert it to data store. And it doesn’t actually change
all that much, but there’s some details. So we grab the image before from
the request and now we just copied into a buffer. So we have a bytes buffer with
the image data inside it. Then we create this thing called
a context, which is an App Engine specific idea that is
associated with the request from the client. And that gives you a
place to do logging information, things like that. But it also gives you
the methods for things like making keys. Sorry, for accessing the
elements of the client’s request. And we use it in order
to put the data inside. Because if we want to associate
the client with the data we’re putting into it. So here we create a new key
using the key of the data we got, and then using the context
and the key, we store the data inside the
data store. And so now we have this hex
string that represents the data that we can use to
get it back later. Having done that as before, we
just redirect to the edit handler, but this time we’re
using this key that we used instead of a file name. Once the request comes in for
the image, we recover the data just by sort of inverting
the process. Calculating what the
context is again. Using the ID from the form value
to recover the data and doing a datastore.get to
pull it out again. Checking errors of course,
all the time. And then once we’ve got the
data, we can just call image.decode on it just
as we did before. So one other major difference
when using App Engine is you can’t just read and write
network sockets. You have to use this thing
called URL fetch API. And to do that, there’s a fair
bit of adjustments you have to make to the way that the
OAuth flow works. But it doesn’t change
all that much. The URL fetch API has this
concept of a transport that let’s you describe how to make
requests through the App Engine infrastructure to general
HTTP interfaces. And all we really have to do
here is take the OAuth transport mechanism and insert
that into the structure for the transport for URL fetch,
which we do by constructing one of these transport
objects here. Having done that, it all just
kind of flows together. And that’s really all
you’ve got to do. So with all that done, you
want to deploy it. What do you have to do? Well, the first thing you have
to do is describe to App Engine what your
app looks like. There’s two parts to that. One is you have to write a
thing called app.yaml. Well, I think it’s a ridiculous
name, but it’s the one everyone knows. And inside that app.yaml file,
you write down a bunch of very simply defined fields, including
your app ID, runtime: go, which
is the big news. And then, say where the handlers
go and how they’re associated with your program. This last line on the script,
it tells the development app server what kind of program
you’re playing with here. Then to assemble the app, what
you need to do is take all your source code and
put it together in one directory hierarchy. And put the app.yaml
file at the root. It’s an easy enough thing
to put the HTML files in the root as well. And then in the subdirectories
of various places you put the source code. We put the two files from
Moustachio into the Moustachio subdirectory. That makes the package
Moustachio. And then we also have to bring
along any third party apps that we needed that were not
part of the App Engine infrastructure by default. So here we need to bring along
the freetype-go and the goauth2 packages. And these get uploaded
from source. And then, once they’re uploaded,
they will be compiled in the cloud and
deployed from a statically linked binary in the cloud. So to test it out once you’ve
got all this put together, you can run the development app
server with a thing that’s very familiar-looking
to App Engine users. You just run the
with the path to your app and it brings
up the local thing. And at that point, you’re
essentially running a program equivalent to what Andrew
showed you. Except the data store is just
local stuff on your machine. Once you’re satisfied with that
story, you can use the app config to deploy the app
into Google’s cloud using the app config update from exactly
the same tree. And once that’s going, you can
access it in the cloud by your app ID I wonder if it’s
going to work. OK, let’s see. No, I’m in the right place. OK. This is a version of Moustachio
at, which is running in Google’s
infrastructure. We’ve made it a little
prettier. We’ve put a little more graphics
in it, but the source code is largely the same as
the one we showed you. We just made it prettier
for the purpose. Now because it’s in App Engine,
we need a different picture, one more
representative. So let’s bring this guy up. And there he is. So we’ve now stored that image
inside the data store inside the Google cloud. We can play with him. I think this guy actually looks
like a French pilot now, so let’s give him the
appropriate French look. Make it a little smaller. Yeah, there we go. And I think we want– that’s a little too much. That’s a little too Spanish. Let’s go French. No, that’s not right. There we go. There he is. There’s our French pilot. And now of course, we want to
share it as we did before, which we do in the usual way. It says, do you want to share
to your Buzz access? And that icon as before is now
telling us that the real Moustachio guy wants to share
your Buzz information or Buzz authentication. You allow access and
it says it’s there. So the image is still there. But now Enough we go to the
Buzz, it should be posted for real and we can click
in it, and all your friends can now see that. So that’s live. [APPLAUSE] ROB PIKE: No demo
effect today. That’s impressive. OK. All right, I think you’re
probably asking at this point if you don’t already,
what about me? Can I do this? The answer’s yes,
I, can do this. Can you do this? Not quite yet, but almost. We’re announcing that Go is
going to be a fully supported language on App Engine. Although, today it’s launching
as an experimental feature, which limits how much you can
use it, number of users you can have, there’s no billing,
experimental is a well-defined term in App Engine space
and Go is definitely an experimental feature. But, it’s real and it works. As of today, the
SDK is publicly available for download. So right now you can go to the
site, grab the SDK and start writing an App Engine app in Go
that will run on your local machine to debug
it and test it. We’re about to start sign-ups
for trusted testers for production into the
cloud for real. If you’re interested
in being a trusted tester, there’s a sign-up. The details are on our blog,
which is now live at And if you want to be
in there, sign up. We’ll be rolling it out over
the next few weeks. Should be fully out within
a few weeks. Most of the App Engine APIs
you’re familiar with from other languages are supported
and the list is growing and they’re all being worked
on actively. And of course, they’re all open
source, so feel free to look at what’s going on there. I would like to make it very
clear that I did not do any of the work on this myself. This was entirely done
by the team. And I’d like to give a special
thanks to the guys who did all the hard work, which is David
Symons, Nigel Tao, Andrew Gerrand, who did talk. And plus we got a lot of help
from the App Engine people themselves and I would
like to thank them. So thanks guys. [APPLAUSE] ROB PIKE: OK, so in conclusion,
we think that Go is actually a great back-end
language for App Engine. And if you think about it, App
Engine is this really easy way to build, deploy scalable apps
into the infrastructure that Google provides. Go is a language for writing
scalable programs. It’s just a perfect fit. And it should be. With this environment you get
an environment for building web apps that’s really easy
and productive to use. It scales really well. The libraries are good. The language is good and getting
better all the time. It’s also a true compiled
language. It’s actually the
first compiled language for App Engine. Because Java goes through JVM. Python in interpreter. If you’ve got some really tight
code you need to do, like the image processing stuff
we showed you or maybe some [? crypto ?] stuff, something like that,
you can actually get right down to the bare metal in Go. And that’s a really nice, new
option to have available. As I said, you can start
using it today. There is the link to
the downloads page. And the source code’s
all online. You can be free to look at it,
comment, send us mail if you see bugs, play with it, maybe
make suggestions if you’re interested in helping us
make it all better. And as of now, there should be
one public App Engine service, which is Moustachio. And I expect to see Moustachio’s
all over the web by the end of business today. So if you want to know more
about Go, is a pretty big site. It’s growing all the time. There’s tutorials, videos, I
think this video should be posted there when
it’s available. There’s code walks and of
course, all the source code’s accessible through the site. The spec is there. And by the way, to Go spec
compared to many other languages is quite concise
and very reasonable. It’s not a huge languages,
it’s quite approachable. We’re moving offices now to a
new, smaller office that we have. And I might have to throw
away my C++ spec because there might not be room
in the new office. The Go blog has got all that,
all the stuff I talked about. And the newest page, the newest
entry, there has all the information that we’ve
talked about today. And then, if you want to hear
more about it or just come talk to us, their office hours
today sort of now until 3:00. We’ll hang around until the end
of this session though. And then tomorrow there’s an
App Engine office hours and we’ll be there with them. And I’m sure that there’s
questions that you’ll have for them as well. So maybe we should start by
taking a few questions. Andrew, do you want
to come back up? We should probably
get the feedback. Is there a question [INAUDIBLE] page? No? ANDREW GERRAND: No, there’s
no [INAUDIBLE]. ROB PIKE: OK. So please use the microphones
for questions if you have any. No? Please come to the mike. Hi. Is there support for
asynchronous calls? ROB PIKE: Asynchronous calls? Well, App Engine, like all
web stuff, is kind of asynchronous. I mean you use a restful API to
construct an asynchronous callback through
the interface. That’s basically how it works. I’m not sure what you’re– ANDREW GERRAND: Do you
mean in a single handler, like making– In Go language. ANDREW GERRAND: So Go supports
some interesting concurrency permitives called go routines
and channels. And they’re fully supported
under the App Engine runtime. And so it’s possible say, if you
want to make multiple App Engine API calls. You can do then in separate Go routines and get the responses. And all of that should work. And the one difference between
running Go locally, in terms of its concurrency permitives
versus deploying to App Engine is that they all run in
a single operating system thread now. So you don’t get any CPU
parallelism, but you do get the full benefits of that
concurrency permitives. ROB PIKE: That restriction
will get lifted. We plan to enable threading. But that’s not to say that all
of your app has to run in a single thread because there
could be multiple instances on different computers. But for each instance, there
will be only a single thread running with all the
Go routines. But you still get all the nice
features of Go to be able to schedule multiple outstanding
requests and stuff like that. It works very nicely. ANDREW GERRAND: It’s the same
limitation the currently applies to the Python
and Java runtimes. ROB PIKE: We really hope to
remove that restriction. Thank you. Can you just review why you
thought the world needed another programming language? And also, why Google are
investing so heavily in it. What’s their motivation other
than yours, may be more academic interest. But theirs
clearly, should be more commercial. How do you– ROB PIKE: Well, it’s a perfectly
good question. And I think we’ve answered it
many times in the past, but I’ll answer it again. To be honest, there’s already a
lot of programming language in the world, but none of them
seem to work well at doing the kind of software that we
develop inside Google. They work well enough, but
they don’t solve all the problems together well. Most of our server software is
written in language like Java and C++, and those languages are
fairly large to write in. They take a lot of tools
to work well. We need distributed compilation
farms. Dependency management, particularly
C++ is a nightmare. And there’s just a bunch of
things that tend to slow down development that are purely
mechanical and determined by the language rather than
necessarily the problem you’re working on. On the other hand, it seemed
to us there was a chance to make nicer language that worked
better with modern hardware that understands
concurrency. parallelism, multi-core
machines, can support networking better, and
things like that. The answer is no, we never
needed a new language, but we thought it would really make
life better if we had one. It’s starting to catch on. I think we found a sweet spot
for the balance between pragmatism and sort
of niceness. Now one of the things I will say
is that when we announced this language, a lot of the
programming community poo-pooed it because they didn’t
understand why you would have a language
that wasn’t theoretically, exciting. And the point was it’s not
theoretically exciting, it’s just very useful. And that pragmatic approach to
the design of the language is really what makes it
so productive. I’ve never used a language
I found more productive. I really think it’s
a nice language. And if you don’t enjoy using
it, then I’m not going to twist your arm about it. But I think you’ll find if you
try it, which a lot of the naysayers have not done, that
it’s actually way more effective than a lot
of people think. [UNINTELLIGIBLE PHRASE] ROB PIKE: Sorry? I do use it and I do like it. ROB PIKE: Please use
the microphone if you want to say something. He does use it and
he does like it. ROB PIKE: OK, thank you. At the front? Will you be supporting the
App Engine pipeline API? And if so, how does that balance
against what Andrew just brought up using separate
Go routines at the client level to process? ANDREW GERRAND: I might have to
ask someone from App Engine to answer that question
for you. We’re sort of covering the
existing core functionality of App Engine first and before sort
of venturing out into the more experimental features. But our goal is certainly
to implement all of the functionality available
to the other runtimes. ROB PIKE: The App Engine
infrastructure is really designed to be pretty much
language neutral because of the way it works. It’s just a matter of writing
the APIs and having them fit well in your language. If there’s an API that is– well, there are a number of
APIs that are missing. But if there is one that you’re
particularly interested in, let us know and we’ll
give it priority to try to get it ready. It’s not a big deal to write a
basic API to talk to one of these facilities. Getting the full rich
feature set could take a lot more time. But it’s not our intention to
deliberately ignore any particular API. It was just what we needed to
be able to launch some demo apps and make it sort of
get off the ground. So we have things like data
store, blob store, the users API, mem cache, the basic set. Mail, right. And they’ll all be there
at some point. Thank you. ANDREW GERRAND: It could be
also in the case of the pipeline system that it works
particularly well in Go because of Go’s concurrency
support. But I don’t know enough about
it to say for sure. ROB PIKE: Yeah, we also have
task queues, which I think is not in both the other
languages. ANDREW GERRAND: There’s
a Java task– That’s in Python too. ROB PIKE: Back? Yeah, thanks. This is a great discussion. I’m new to Go and one
of the concerns I have is library support. I was at NodeConf last week and
one thing that was really interesting was they talked
about dom on the server side and that opens up for a lot of
libraries that are available or previously only available
on the client side with JavaScript. So can you just talk about what
the true state of library support is, and are there any
future directions, like for example, being able to use
Python or Java libraries inside of App Engine
that talk with Go? ANDREW GERRAND: So in terms of
Go in general, library support is improving at a
very rapid pace. There are many, many external
libraries listed on our package dashboard, which is
linked to from And those are for accessing
many different types of databases or [INAUDIBLE] support and so on. And graphics programming,
et cetera. But as far as linking non-Go
libraries to Go code in App Engine, that’s definitely
an unknown. It makes the whole process of
sandboxing, the process for security purposes, much,
much more complicated. And that’s why you can only
upload pure Go libraries to the Go App Engine runtime. But as far as being able to use
more and more libraries across languages, I mean it’s
something that we’d definitely be interesting in doing, but I
don’t have anything to say about it at this stage. I think one of the other
questions I have was want to echo what the previous gentleman
said, is that Go is great and I could see lots of
uses, but I also have lots of Java code that I would like to
have Go act as kind of the dispatcher around. But that there’s no value to
convert from one language to another in many cases. But now my other question is,
how do you handle things like, where you have multiple
requests fired off and something as simple if none of
the requests return a certain amount of time, return a
default, I didn’t really see an example of that. But I’d really love to see how
you handle a complex kind of a [UNINTELLIGIBLE] kind of thing with. Because I suspect that you have
some sort of conditional semantics around
how to do that. ROB PIKE: Yeah, there’s a lot
of stuff in Go’s concurrency support that makes that
stuff work well. There’s actually, unrelated
to this stuff, there’s stuff on the blog. There’s been explanations of
timeouts in our use channels to coordinate things. The talk we gave last year
actually included a task queue manager presentation, which
is relevant to that. It’s sort of hard to do it
spontaneously to explain it. But basically, you use go
routines and channels to construct the architecture and
you can time things out and it’s actually very, very easy. It’s arguably much easier than
using asynchronous call backs. But that could be largely
a matter of taste. But it’s certainly easy to do
and I think a lot easier than certainly in Java. Back? Hi, yeah. How has memory manager
been done with Go? ROB PIKE: Memory management,
it’s a fully garbage collected language. OK, great. ROB PIKE: It’s actually better
than that though, because unlike most garbage collected
languages, you can have internal pointers, which means
that you can have a– for instance, if you have a
structure and you want to store a buffer, you can put
the buffer inside the structure and do only one
allocation for it. Which means only the overhead of
one collector as opposed to having to do a second allocation
to allocate the buffer offline. So it’s like C in the sense you
have complete control over memory layout, but it’s a
fully garbage collected language nonetheless. And it’s also completely safe,
memory safe, type safe. So it’s an interesting
data point. That brings me to my question. Modern machines depend a lot
on the CPU caches for performance. And garbage collection
algorithms tend to lose cache very rapidly. I was wondering how you handled that engineering trade-off? ROB PIKE: Well I sort of half
answered the question before. The thing is that if you– a garbage collector
isn’t free. If you have a garbage collector,
there’s an overhead to run it. There’s no question
about that. There’s also an overhead if
you’re a programmer and you don’t have a garbage collected
language to manage the memory. And we decided to make the
trade-off away from putting a load on the program, which means
the garbage collector has to be there. And also, for concurrent
languages, garbage collector’s a must because you can’t be
dealing with multiple threads, dealing with multiple bits
of memory and figuring out who owns what. But in return for that,
you have complete control over memory. So if you understand that it’s
a garbage collected language, which you should if
you’re writing performance intensive code. And you understand what the
costs are for allocating and freeing memory, it’s possible to
construct your app so that it does only the minimum amount
of allocation beyond what’s necessary. And so it ends up being in
principle, as efficient as say, a C program. And you have to know what you’re
doing to get that. But at least it’s possible. Whereas, in a language like Java
where you just have to let the system deal with the
memory, it’s much harder to control allocation. For instance, if you get a
client request, in principle, you can put everything you need
into a single allocation and let the thing handle that. It’s not easy to do that. But in principle. you could do a
single allocation per client and make it work. I mean I think that’d be
really a challenge. But you could do it. Whereas, it’s not even possible
in most garbage collected languages. So we try to help you out. ANDREW GERRAND: Somebody
at the back. GWT and Android would be nice
places to see Go show up next. ROB PIKE: Yes, they would. Any time line plans,
skunk works? ROB PIKE: Can’t talk about it. So can you talk a little but
more about the compiler that’s used on Google App Engine? Is it open source also? ROB PIKE: It’s open
source, yes. The compiler’s open source. It is the bizarrely
named 6g compiler. So it’s a 64 bit, x86 compiler
written in C, but it compiles Go code into– after the linker, a statically
linked binary that just gets deployed as the binary. So you upload your source
and it gets compiled by this cloud. If you go to you’ll
find the links to the open source place. It’s And the compiler’s there. So fully open sourced compiler,
live, everything . It’s all open source. So another question
about scripting. We’d like to see it run in
the browser too, Go. ROB PIKE: So would we. OK. ROB PIKE: Believe me,
we’re on your side. We really are. Any more questions? Has our surprise arrived? Yes? ANDREW GERRAND: Yeah. [UNINTELLIGIBLE PHRASE] ROB PIKE: Was that a yes? OK. ANDREW GERRAND: Was that–
it was a yes. ROB PIKE: Was a yes. OK, as you leave, don’t forget
to talk about what you heard here on Twitter. Check the feedback forms out. And also, you may find there’s
a collection of gophers waiting outside for everybody. [APPLAUSE] ROB PIKE: Thanks everybody
for coming. Have fun.

45 thoughts on “Google I/O 2011: Writing Web Apps in Go

  1. YouTube has *massive* bandwidth problems. Listening to this video like I have a 56k dial-up modem connection πŸ™

  2. I'm new to Go, since about a week ago. Useful video.

    I know Go has changed quite a bit over the past 12 months, so I wonder how relevant earlier videos are for people (like me) getting up to speed with the language. E.g. the 2010 I-O material.

  3. @FlameHue so what you know about computing that make you smarter that those dumbest people?

    So C and C++ is developed by bell labs which is a subsidiary of AT&T (that time), isn't AT&T a corporation too? that make C and C++ a corporate programming language just like Go, Java, etc…

    whay definition of corporate programming language anyway?

  4. @formicFabricant P1: My perspective comes from the 3D world, graphics and games, which most ppl making these languages seem to pay no mind. If you are trying to write a simple physics engine or processing a bunch of vertex calculations,.. this need for custom operators would become very clear.

    Vectors should be used like generic types, just as they are in HLSL or GLSL because in concept generics are vectors too (float x = Vector1D, float x,y = Vector2D, float x,y,z = Vector3D).

  5. @formicFabricant P2: Now try to add that Vector3D just like you would a Vector1D(aka float)…? Well you can't in Go and that is a problem if you want consistent clean and clear to understand "mathematical" code (aka data crunching code).

    Rly there are three needed things here for proper vector processing. Strucks, Custom Operators and multiple overloads. Go only has strucks. Multiple overloads would be for thinks like dot products (vec.Dot() or vec.Dot(vec2)).

  6. @formicFabricant P3: Go's focus seems to only be about services. C/C++ or C# have much bigger perspectives as all of them are used in devices, games, robotics, services, graphics, GUI's, ect.

    You have to use special tools in things like Java to get custom operators and I don't want to see the same thing happen to Go as I like some of its concepts but will never use it unless it has custom operators or multiple overloads.

  7. Goweb (on Google Code) is a framework that makes building APIs and RESTful websites very easy… including Ruby on Rails style routing etc. Check it out and please contribute with ideas and code.

  8. @formicFabricant That's a good idea, tnx. Although Native Client will come out at some point and C++ or .NET C# through Mono will be on there, so i'll just stick with that as it does what I need and runs on everything I wan't.

    Also I notice you can't return the starting method bracket. I very much dislike that standard. Go is focused on pleasing Java and Javascript devs… I don't think its for me.

  9. @ochgottnochma Ubuntu or derivatives:
    sudo add-apt-repository ppa:gophers/go
    sudo apt-get update
    sudo apt-get install golang

  10. The most popular genre is romance writing. Come join us for a series of classes where I will guide you on how to write your own romantic novel or novelette.

    Come join us for a series of classes where I will guide you on how to write your own romantic novel or novelette.

    Get More Details on

  11. It's an annual event of Google where developers from around the world come together to attend technical sessions. These sessions usually focus on mobile, web and enterprise app development using Google's own technologies.

    It's also where the lucky participants get various freebies like the latest "Google-made" phones, tablets and other such gadgets. πŸ™‚

  12. I'm curious why google is so into capitalization (v8's internals, go, etc…) is this to avoid keywords?.. I dont get it, that's a major turnoff really, sounds dumb I know but it looks really bad

  13. Please Google, make all those videos available in 1080p. You control the production of those videos from A to Z, it needs to be polished and refined if you want to be serious about promoting Go.

    Excellent video though.

  14. Google IO is a developer-focused conference held by Google. You could have used Google to find the answer, so yeah it was a stupid question.

  15. Holy f*k, it's Rob Pike. If you don't know who he is, don't tell me. I'll be angry.

    By the way, you do not need overloading and custom ops. Good god, did you never use a language with a non-oo paradigm, like Scheme, or C? I could understand not knowing Scheme, but you have to know C, at least the basics. Do you need custom ops? No, call strcmp, you moron.

Leave a Reply

Your email address will not be published. Required fields are marked *