Client-side-only Javascript Amazon S3 CMS

Amazon S3 just released a feature that allows you to host a website on S3. I’ve been wanting this feature for years and now that it’s here I can’t keep my mind off of building a CMS on top of it. There’s just so many interesting challenges to figure out and technical coolness.

Javascript Client

The idea is to build a CMS that is 100% HTML and Javascript, hosted from S3 itself. You would upload the initial files for the admin, set an appropriate bucket policy, and set up the website features on your bucket (or more likely when I’m further along put your Amazon creds into an online upload form and allow a little web service to do it for you). Then you would navigate to https://yourbucket.s3.amazonaws.com/admin/index.html and login to your hosted javascript admin. Logging would keep the key and secret in memory and you’d be able to add, delete, update, and otherwise work with your website on S3, from S3.

Benefits of Client-side Javascript and S3

There are some really cool aspects to a 100% client-side Javascript application and hosting on S3.

The biggest IMO probably being pluginability. Most people can write Javascript and many know jQuery, so writing plugins that utilized both to extend the interface, respond to actions, and otherwise modify the system would be easy and allow for very customized web admins and sites. You also have a built-in REST API that you can store anything to. If you want to add TODOs to the cms, you can store them in /api/todos/ immediately without having to create a new database table and worry about setup. You can also update your TODO plugin later to include priority without needing to update tables.

There is also the draw of high-traffic websites not hitting any servers you have to worry about. Let Amazon deal with the hosting and meet their “99.99% availability”. You can handle massive load. And it’s cost-effective. And stable/redundant, you don’t need backups. And you can front the whole thing with CloudFront to deliver your pages at the closest locations possible (you’ll have to invalidate pages that get updated though).

Drawbacks

It’s hard to have dynamic websites without server-side logic. You can’t accept comments on your posts. You can’t resize images on the fly or even at upload time (well, with canvas, but that’s something to research more on viability). You can’t send out emails. And you can’t utilize cross-site resources or APIs (such as CloudFront or Akismet). However, if this cms turns out well and people use it, I can create a service that will allow all of these things for registered users.

Previous Work

When looking around for an S3 Javascript client I found that back as far as 2006 people were thinking about building Javascript applications hosted from S3. It didn’t really seem to take off, but there’s a lot more work that’s been done on S3 since then. It might be more interesting now. Especially if some general hurdles are figured out so that your site isn’t hacked, spammed, and ruined.

Security

My first and biggest concern for something like this is handling security. How does one have an all client-side application and still allow secure login and user management? The first thing that comes to mind is the obvious: why not have the user enter their Amazon key and secret to login? That’s a good idea. At least, it will work. Using a the bucket policy file we can specify an area of the site that is private for reads and writes which can store website data (all of it will be private for writes). But if this is going to be something I use, I don’t want to go look up my amazon key and secret every time I need to log in. So I brainstormed other solutions.

AES

I need 2-way encryption that will allow me to store the amazon key and secret in a public location. Public because you can’t access it before you’re logged in now can you. The most secure 2-way encryption I found was AES which is used in government and many other places. It is quite secure and wonderfully Movable Type has implementations of AES and SHA1 and lots of other stuff in Javascript.

A user will register with a username (or email), password, amazon key, and amazon secret. The password will be used to encrypt a string of “username:key:secret” and saved to a public file by the sha1 of the username under the private folder api/auth/. Having the folder private will prevent people from seeing what is in it. You’d have to guess the sha1 to even get the file, and then you’d have to know the password to decrypt it. After registering, a user can login with their username and password. The username will be sha1’ed and the user file will be loaded (if it exists). Then the password will be used for deciphering the contained hash and the key and secret put safely away inside a Javascript closure in memory.

This seems quite secure. For one, the username and password never even go over the wire, so even if we weren’t going to use SSL (which we are) the only thing a man-in-the-middle would see is the encrypted AES hash. If anyone has better ideas or sees holes in this I’d love to fix it up more securely.

Features

My other main concern is whether one could really support many features of a common web cms with only client-side logic. I’ve been quite surprised at how many features I’ve thought could work with simple Javascript. Here is a list of those I’ve thought of so far:

Templates: pages, content, templates, etc. could be stored in the api and whenever you update a page, jQuery can put the content into the template and save the whole file out to its location. Templating would be similar to other systems except that when you change the theme or update the template it will have to re-process each file that uses the template individually.

Plugins: plugins would work by attaching onto hooks or listeners throughout the system or adding new editable-field types. They could add HTML to the admin to include new buttons or menus. And their javascript could be concatenated and minified into one plugin file that gets loaded at runtime whenever you activate a new plugin, then you have one small file to load when you log in.

File upload: it turns out you can sign a form a certain way in order to upload files to S3. This is great since you can’t load (in all browsers) files into Javascript first and then sign them and put them to S3. So lage images, video files, etc. can be done without passing through Javascript.

Blog support: you can easily do a blog by recompiling the home-page template whenever you publish a new post. You could even recompile an RSS feed client-side and put that to S3 at the same time.

Shared content: menus, footer content, and other shared content could work, just like when editing a template. Any page that piece of content is shown on will need to be recompiled, however this could easily happen in the background with a little progress indicator while you continue editing or modifying your website. If you try and close up shop before it is finished the admin can warn you that leaving the page will leave your site partially undone.

Additional users: you could either 1. give access to another user’s amazon account to edit your site and let them register using their amazon key/secret, or 2. you could add more users with username/password using your amazon key/secret which will create their “auth file” using those creds. They would never need to touch your key/secret, although they could find it out, so don’t use this method with people you don’t trust.

Remember me: you could store username/password in a cookie for “remember me” functionality. If you do this then maybe using the sha1 of the password to encrypt the AES cypher and storing the sha1 password in the cookie would be a better idea. Keep that thing as safe as possible. Though unsafe plugins could take either version out of the cookie and use it to obtain your key/secret. Maybe use a separate path for login since cookies can restrict on path? Something to think through more.

Code highlighting: code or syntax highlighting can be done at save-time using the Google Code Prettifier project or another Javascript-based project of similar sort. It would just be a plugin.

Content: you can have all the same content other CMSes allow: html, js, css, flash, images, galleries, xml, etc. E.g. use a Javascript plugin to create the XML that your Flash photo gallery needs to run.

Versioning: you can use S3’s built-in versioning to keep every version of an object saved to the bucket and revert back to an old version if desired, or undo a delete.

Open Source

This is something crying out for open source. It has been really fun to start and should be really fun to continue working on. I’ve put the beginnings up on github calling it Static Site, since it is hosted on S3. Should I call it S2? :) I also have staticsite.org hosted from my own S3 bucket which will be using the CMS as it comes along. I hope to steal a moment here and there from my busy work schedule when I need a break in the evenings to keep working on it.

What do you think? Would you use it?

15 Responses to “Client-side-only Javascript Amazon S3 CMS”

  1. l.m.orchard Says:

    Author of S3Ajax, here. Funny thing is I just tweeted about something like this recently:

    http://twitter.com/#!/lmorchard/status/44224437834350594

    JS is more capable than ever, S3 just added better CNAME and index.html features, and Mozilla is exploring installable web apps in the browser. Seems like a perfect storm for an all-browser CMS app.

    Interesting way of converting key/secret into a more humane username/password with the public storage and AES encryption. Gives me the creeps a little, sending my AWS credentials off my machine in some form, but I can’t think of any flaws.

    What I did back in 2006 was use some hacks from Dojo to stash the key/secret in some form of local storage that would never go over the network. Back then, that used Flash or some other hack.

    Today, we actually have window.localStorage in most browsers, which is what I would use if I ever got back around to playing with this.

    But, your encrypted credentials scheme has the benefit of being more usable on a mobile device, many of which now have Javascript runtimes that are quite capable of participating in an all-client-side CMS now.

    Interesting stuff!

  2. l.m.orchard Says:

    Oh, one other thing: If you’re asking for username/password to access & decrypt the key/secret – might as well derive the api/auth filename from SHA1 hash of username:password to make it even less guessable than just username

  3. l.m.orchard Says:

    (Of course, that means you’ll have some old junk auth files lying around if I forget my password, because I’ll never be able to access that auth file again… and if I re-register, then I’ll have my AWS credentials in multiple encrypted files… hmm…)

  4. l.m.orchard Says:

    Oh, and while you’re looking at JS on the CMS side for producing the content to ship to S3 – don’t forget about JS on the browsers of people visiting the site.

    While it’s good to keep progressive enhancement and graceful degradation in mind, you could make a visitor’s browser do some of the work for you.

    Publish nice & clean HTML for a page or an article. But, include common elements such as sidebars with recent archive posts by way of static JSON data shared between pages and render it in JS on the visitor’s browser.

    That way, your browser doesn’t have to update a baked-in sidebar on every page in your site when you post new content. It just has to update some shared JSON indexes.

  5. Jacob Wright Says:

    Some great ideas. Want to help? You know you want to. Would be fun…

  6. Jacob Wright Says:

    Would be interesting to have a site where every page is just the content for the page and it loads its own template file from the server. Would that enhance SEO??

  7. Peter De Keer Says:

    I would love this idea too.
    I tweeted about it (http://twitter.com/#!/tixiv/status/75649712191062018)

    If it’s a mozilla installeble app or Google App, doesn’t really matter.

    If this hits off in any way i’m willing to participate!
    Watching the code on github too

  8. Alex Says:

    As I understand, anyone would be able to visit such a CMS, leave posts and the like and the CMS would be triggering some functionality that requires the Amazons credentials. If the credentials are going to be sent straight to the visitor’s browser then what would stop them from stealing the credentials?

  9. Antti Haverinen Says:

    Dear Jacob,

    Gladly that I came across your site. I have been thinking of the same. I am not a programmer but I would be ready to contribute other ways. What is the status of the project?

    Sincerely,

    Antti

  10. Dominic Watson Says:

    Just thinking the same thoughts and sitting down to write something rough as a Google Chrome extension. First stab would be to have the user set up their own Amazon S3 bucket, entering their credentials in the Chrome extension settings page (saved in local storage).

    Perhaps build the engine around the Jekyll CMS concept or some such (save reinventing wheels).

  11. Richard Says:

    Agreed – I’ve had similar thoughts. I’ve always hated the idea of big J2EE engines or misc CGI scripts, and template files/languages, and database APIs.

    RESTful S3, Javascript, and HTML seems like the simplest, coolest way to build a fair percentage of sites out there.

    BTW, regarding Wikis – A tiddlywiki via S3 would be very neat.

  12. James D Says:

    HELL YES I would use this. Perhaps if it also had Markdown input too?

    Imma gonna fork it and have a go adding Markdown if you don’t mind?

    ^JD

  13. Gretchen Vanderlaan Says:

    Amazon is great, but may also end up as expensive. What if you have few TB’s of data, that you want to access remotely, with Amazon (if I remember correctly) you need to pay few hundred bucks monthly just to have it there well it’s not surprise, however in such more extreme case, colocation of your own data storage server ends up cheaper

  14. Bill Says:

    @Gretchen
    Expensive!? All relative.
    It’s about ease of use for not only us techies but those for whom you allow to modify their ‘cms’

    @Jacob
    Nice succinct writeup – as i came here, based on that particular search “cms site on amazon s3” ;) yours came up!

    thx.

  15. Janaka Says:

    Nice one! Have been thinking about this since 2009.