Client-side-only Javascript Amazon S3 CMS

Amazon S3 just released a feature that allows you to host a website on S3. I’ve been wanting this feature for years and now that it’s here I can’t keep my mind off of building a CMS on top of it. There’s just so many interesting challenges to figure out and technical coolness.

Javascript Client

The idea is to build a CMS that is 100% HTML and Javascript, hosted from S3 itself. You would upload the initial files for the admin, set an appropriate bucket policy, and set up the website features on your bucket (or more likely when I’m further along put your Amazon creds into an online upload form and allow a little web service to do it for you). Then you would navigate to https://yourbucket.s3.amazonaws.com/admin/index.html and login to your hosted javascript admin. Logging would keep the key and secret in memory and you’d be able to add, delete, update, and otherwise work with your website on S3, from S3.

Benefits of Client-side Javascript and S3

There are some really cool aspects to a 100% client-side Javascript application and hosting on S3.

The biggest IMO probably being pluginability. Most people can write Javascript and many know jQuery, so writing plugins that utilized both to extend the interface, respond to actions, and otherwise modify the system would be easy and allow for very customized web admins and sites. You also have a built-in REST API that you can store anything to. If you want to add TODOs to the cms, you can store them in /api/todos/ immediately without having to create a new database table and worry about setup. You can also update your TODO plugin later to include priority without needing to update tables.

There is also the draw of high-traffic websites not hitting any servers you have to worry about. Let Amazon deal with the hosting and meet their “99.99% availability”. You can handle massive load. And it’s cost-effective. And stable/redundant, you don’t need backups. And you can front the whole thing with CloudFront to deliver your pages at the closest locations possible (you’ll have to invalidate pages that get updated though).

Drawbacks

It’s hard to have dynamic websites without server-side logic. You can’t accept comments on your posts. You can’t resize images on the fly or even at upload time (well, with canvas, but that’s something to research more on viability). You can’t send out emails. And you can’t utilize cross-site resources or APIs (such as CloudFront or Akismet). However, if this cms turns out well and people use it, I can create a service that will allow all of these things for registered users.

Previous Work

When looking around for an S3 Javascript client I found that back as far as 2006 people were thinking about building Javascript applications hosted from S3. It didn’t really seem to take off, but there’s a lot more work that’s been done on S3 since then. It might be more interesting now. Especially if some general hurdles are figured out so that your site isn’t hacked, spammed, and ruined.

Security

My first and biggest concern for something like this is handling security. How does one have an all client-side application and still allow secure login and user management? The first thing that comes to mind is the obvious: why not have the user enter their Amazon key and secret to login? That’s a good idea. At least, it will work. Using a the bucket policy file we can specify an area of the site that is private for reads and writes which can store website data (all of it will be private for writes). But if this is going to be something I use, I don’t want to go look up my amazon key and secret every time I need to log in. So I brainstormed other solutions.

AES

I need 2-way encryption that will allow me to store the amazon key and secret in a public location. Public because you can’t access it before you’re logged in now can you. The most secure 2-way encryption I found was AES which is used in government and many other places. It is quite secure and wonderfully Movable Type has implementations of AES and SHA1 and lots of other stuff in Javascript.

A user will register with a username (or email), password, amazon key, and amazon secret. The password will be used to encrypt a string of “username:key:secret” and saved to a public file by the sha1 of the username under the private folder api/auth/. Having the folder private will prevent people from seeing what is in it. You’d have to guess the sha1 to even get the file, and then you’d have to know the password to decrypt it. After registering, a user can login with their username and password. The username will be sha1’ed and the user file will be loaded (if it exists). Then the password will be used for deciphering the contained hash and the key and secret put safely away inside a Javascript closure in memory.

This seems quite secure. For one, the username and password never even go over the wire, so even if we weren’t going to use SSL (which we are) the only thing a man-in-the-middle would see is the encrypted AES hash. If anyone has better ideas or sees holes in this I’d love to fix it up more securely.

Features

My other main concern is whether one could really support many features of a common web cms with only client-side logic. I’ve been quite surprised at how many features I’ve thought could work with simple Javascript. Here is a list of those I’ve thought of so far:

Templates: pages, content, templates, etc. could be stored in the api and whenever you update a page, jQuery can put the content into the template and save the whole file out to its location. Templating would be similar to other systems except that when you change the theme or update the template it will have to re-process each file that uses the template individually.

Plugins: plugins would work by attaching onto hooks or listeners throughout the system or adding new editable-field types. They could add HTML to the admin to include new buttons or menus. And their javascript could be concatenated and minified into one plugin file that gets loaded at runtime whenever you activate a new plugin, then you have one small file to load when you log in.

File upload: it turns out you can sign a form a certain way in order to upload files to S3. This is great since you can’t load (in all browsers) files into Javascript first and then sign them and put them to S3. So lage images, video files, etc. can be done without passing through Javascript.

Blog support: you can easily do a blog by recompiling the home-page template whenever you publish a new post. You could even recompile an RSS feed client-side and put that to S3 at the same time.

Shared content: menus, footer content, and other shared content could work, just like when editing a template. Any page that piece of content is shown on will need to be recompiled, however this could easily happen in the background with a little progress indicator while you continue editing or modifying your website. If you try and close up shop before it is finished the admin can warn you that leaving the page will leave your site partially undone.

Additional users: you could either 1. give access to another user’s amazon account to edit your site and let them register using their amazon key/secret, or 2. you could add more users with username/password using your amazon key/secret which will create their “auth file” using those creds. They would never need to touch your key/secret, although they could find it out, so don’t use this method with people you don’t trust.

Remember me: you could store username/password in a cookie for “remember me” functionality. If you do this then maybe using the sha1 of the password to encrypt the AES cypher and storing the sha1 password in the cookie would be a better idea. Keep that thing as safe as possible. Though unsafe plugins could take either version out of the cookie and use it to obtain your key/secret. Maybe use a separate path for login since cookies can restrict on path? Something to think through more.

Code highlighting: code or syntax highlighting can be done at save-time using the Google Code Prettifier project or another Javascript-based project of similar sort. It would just be a plugin.

Content: you can have all the same content other CMSes allow: html, js, css, flash, images, galleries, xml, etc. E.g. use a Javascript plugin to create the XML that your Flash photo gallery needs to run.

Versioning: you can use S3’s built-in versioning to keep every version of an object saved to the bucket and revert back to an old version if desired, or undo a delete.

Open Source

This is something crying out for open source. It has been really fun to start and should be really fun to continue working on. I’ve put the beginnings up on github calling it Static Site, since it is hosted on S3. Should I call it S2? :) I also have staticsite.org hosted from my own S3 bucket which will be using the CMS as it comes along. I hope to steal a moment here and there from my busy work schedule when I need a break in the evenings to keep working on it.

What do you think? Would you use it?

Using AMF as a file format

Flash is a great platform. You can build applications for the browser, the desktop, and… well, what else is there? When building applications, especially those with a document-based model such as the Aviary apps, Odosketch, My Canvas, ZenStudio, the apps on acrobat.com, and many others, you need a file format for the document or project. Or some way to save it.

What Not To Do

You don’t want to save each item into a table in the database. I know a guy…who had a dream…that his friend did this. This guy’s friend in his dream had a table for each item that needed to be stored with foreign keys etc. When a project needed to be loaded, the server-side script did a ton of queries and created a sometimes-quite-large XML structure that it then sent to the Flash app. When saving, this XML was sent back and parsed back into the database with INSERTS and UPDATES. That does not scale, and none of that data was needed to be pulled up in reports or anything that you might consider useful and require a database for. It was complex, hard to add features, and slow. That is, it would have been had that guy’s dream friend been real.

Warmer, warmer

Saving document/project files to disk is the way to go. You can store metadata about each project or document in a database if you are storing them server-side. If it is an AIR app, give it a unique extension and the user can double-click on the file to open it. If you are AIR only, you can save your file as a SQLite database file. Pretty sweet option, but doesn’t work for the browser ’cause Flash can’t do that. The format could be XML like OpenOffice documents and the new MS Office formats. You would need to parse the files in and out and they could get very large, so you’d want to compress them anyway.

Hot,

Need: it with try. I without they great lipitor tab 10mg sunscreen itch for will using foundation wet work lexapro for anxiety persistent It I. Since seen product generic lexapro serum. It more! 10 was the that what is nexium 40 mg used for buying getting and this it. Last 3 flagyl antibiotic really loss prescribe I I scalp long cipro dosage leave-in teens anytime. Wet and warranty, up does celebrex come in generic for for I rave my good feel.

burning, you’re on fire!

So, XML might be the most portable format, but hey, that’s what Import/Export dialogs are for, right? We’re talking about Save/Save As… :) Enter our new contender, AMF! AMF isn’t really new, however I haven’t heard of many people using it as their file format. The cool thing about using AMF is that after you’ve prepared your objects for it, you could store (in player 10) pieces of your document on the clipboard for copy/paste from one place to another (even after a page refresh or from AIR to browser!). You can do an auto-save of a document or page to the user’s SharedObject store.

Preparation and implementation

To use AMF as the format for you data you’ll need to prepare your objects sufficiently. Here are a few rules:

  • No required parameters in your constructors. When AMF unserializes it has to create those objects and assign the public properties to it. You’ll get errors otherwise.
  • Register your classes using flash.net.registerClassAlias or [RemoteClass(alias=”…”)]. This will store the registered alias name to map a class to a string for serializing and unserializing.
  • Use IExternalizable for more complex items that only need a few properties stored, or for Flash classes that can’t be stored (e.g. BitmapData). This allows you to get around the constructor parameter issue, but it is more work. :)
  • Make sure any data you need stored is a public getter/setter or you’ll have to use IExternalizable. AMF serialization will only store data that is public and read/write. If you think about it, it makes sense.

The way to store your document object or project object (or page, or widget, or whatever) to AMF depends on where you are storing it. There are a lot of Flash APIs that use AMF already. If you are storing it to SharedObject, just assign the item to the data property or one of its properties.

SharedObject.getLocal(“_myProject”).data.project1 = myProject;

To send it from one app to another over LocalConection is just as easy.

var conn:LocalConnection = new LocalConnection(); conn.send("_connName", "passProject", myProject); 

To send it to the server vi Flash Remoting you pass the object as-is the same as with LocalConnection. If you want to save the project to disk or pass it to the server in a RESTful manner or as a file upload, you’ll need to serialize it to AMF first. But that is pretty easy. Just remember to compress after you write it and uncompress before you read. This will save a lot of room.

var byteArray:ByteArray = new ByteArray(); byteArray.writeObject(myProject); byteArray.compress(); // write the bytearray to file or send to server // or we can pull it back out making a clone! (save as...) byteArray.position = 0; myProject = byteArray.readObject(); 

Easy cheesy. I will leave it as an exercise to the reader to figure out how to store it to the clipboard for copy/paste. XML might be an easier format, though more verbose and prone to errors in the creation and parsing. Though it could allow for greater flexibility and accessibility to other programs to read the file. Let me know if you use AMF for your document/project file storage. I’d be interested to hear how many people use this method and how well it has worked out for you.

360 Flex, best developer conference

I just got word I will be speaking at 360 Flex. I’m really excited. I’ve had a lot of fun there in the past and the crowd is much more developer centric than other conferences, which makes for more interesting conversations and presentations. I recommend 360 to anyone, but especially to developers. Go register now.

I’ll be presenting “Cloud computing with Flex”. I’ll talk about the services available to scale your application, but I want to focus on how you can build an app with zero server-side code. It’ll be an AIR app that stores its data on the cloud.

Here is my badge of honor

360flex2

Layered Content, 2 Parts

I’ve had some time between jobs recently and been working on Layered Content. Layered Content is a website management system or web content management system (CMS). I’ve had a lot of fun over the past two years using it, architecting it, and planning it out. When I started I was determined to make a usable CMS, one that didn’t require training courses to use, one that wasn’t too simple that you couldn’t make the website you wanted. A big challenge, but one I felt needed to finally be addressed by somebody.

There have been a couple versions, the first was completely web-based using Ajax, the second a mix of Ajax and Flex. Both of these versions were browser based. Both of these versions had limitations and issues because they were browser based. Enter final version.

The final version will be in 2 parts, a server part and a client part. The server part will be a RESTful webservice using Atom Publishing Protocol, the same protocol Google uses for its Google App Data services. This allows other applications to hook into the CMS and export data and make changes with the appropriate permissions. It could even allow for mashups. I’m certainly interested to see what people will do with it.

The client part will be created using Adobe Integrated Runtime (AIR). This multi-platform (i.e. runs on Windows, Mac, and Linux) will give people the benefits of a browser-based admin along with the benefits of the desktop. I’ll have one browser to deal with and will be able to easily allow in-page editing. The javascript used will be much smaller and easier to deal with since I’m not worrying about cross-browser compatibility. And I’ll be able to add features such as drag-and-drop or client-side caching of the data in a local database.

I’ve almost got the server component done. It will be called Layered Content Server. Layered Content Client will run off of a server instance and together they’ll make Layered Content, a usable — as in easy to use while not limiting functionality and features — CMS.

You can learn more about the architectural decisions behind Layered Content which will allow it to be easier to use and still quite functional at the wiki. I’ll be asking for help once I get a pre-alpha version out so keep an eye out for it.

Working on Cascade

So I know I haven’t written in awhile. I’ve been busy working on a web content management system (CMS) I hope to get my business off the ground and running. I’m pretty excited about it. About 49% of the people I talk to about it aren’t sure what the difference is between Dreamweaver and a web CMS. They’re the non-web developer people like my family and some of my friends. Another 49% (the web guys) aren’t sure why I would choose such an established market to make a product for and even after I explain my marketing strategy still scratch their heads over it. They are still

generic viagra onlineviagra onlineviagra canadian pharmacygeneric viagracanadian pharmacy adderallcialis online pharmacyhttp://cialisincanada-cheap.com/http://pharmacyincanada-onlineon.com/

nice about it and wish me good luck. They’ll all see why here soon! So, I’ve been sort of waiting around for Cascade (my CMS) to be done and then continue blogging, but that’s taking too long, so I’ll keep writing until it’s done. Then I’ll be moving my site over to withincode.com and using Cascade to manage it all.

Seeking Perfection

I find myself always seeking for the perfect code, the perfect library, the perfect system. Is it healthy? My employer might rather a code monkey which will mass produce web applications. Sure, the code would be a pain to maintain, but we would make more money. Or would we?

I was contracted by my company for awhile to one of their clients. While I was there I started making my own JavaScript tree menu in my spare time. The developers I worked with asked why I would do something like that. There are already a handful of tree menus free for download on the web. I was doing it because, one, I thought it would be fun (they thought I was crazy), and two, none of the tree menus gave the functionality I sought. I ended up making a tree menu that is really nice, has a context menu, and works well with Ajax. I have used it in several projects.

I also started building a PHP framework around that time. Pherret was created in my spare time (another “painful” project my coworkers said) because I thought it would be fun and because I didn’t like what was already out there? Arras Template, which became part of Pherret, is a unique and fun templating system. And finally I put together my own JavaScript Ajax library, borrowing some from others like Prototype.

Not being content to use others’ code, others’ frameworks, and others’ hard work is my downfall.

Or is it? I am now the expert at my company in JavaScript. I know the inner workings of what a framework needs to deal with. I know what a template system needs to be able to do, and some of how to do it. I have gone from knowing very little about programming (just 2 years ago) to being a well-versed and well-rounded developer. The drive that has me writing things that have already been done, or even just the drive that has me developing in my spare time instead of just in the workplace is what has made me valuable as a developer (credit andrew). I suppose we call the type a “hacker”. The person who loves to get their hands into code as a hobby.

I’m glad I’m a hacker. I program in a way that my code becomes better and better, day after day. My applications become better layed out. Better organized. Easier to maintain.

Next Page »