Thursday, April 7, 2011

More thoughts on IndexedDB and Web SQL Database

I've been reading the latest thread discussing the W3C's decision to deprecate Web SQL Database, and it isn't pretty. Key take-away? Web developers don't want IndexedDB, web browser writers don't want Web SQL Database. The major sticking point concerning the latter is the dependency of the spec on SQLite.

Some random thoughts, in no particular order:
  1. SQL is the storage lingua-franca of the World Wide Web, for better or worse. Back-end code is generally discouraged from using custom storage solutions, with access to a database with a SQL API more common, and SQL is even built-in to systems like ColdFusion (not that that's an endorsement...) as an alternative language for accessing internal data structures. No serious web developer can work for very long without a basic understanding of SQL.
  2. SQL's use has not been without criticism. See "NoSQL".
  3. SQL is the only "standard" language for relational database access that has widespread familiarity. Alternatives, such as the proposed RelationalDB Javascript API, are entirely new, unfamiliar, APIs that developers may eschew by default. They are also re-inventions of the wheel.
  4. It's very hard to implement a good SQL engine even if you have a good library of low level table management code.
  5. Abstraction is nice in theory, but has a tendency to be overdone, leading to overly complex, bloated, code that requires huge amounts of memory and CPU usage.
  6. The W3C isn't very good at putting together useful standards. CSS is a disaster - poorly defined concepts leading to multiple ways to "do" the same thing, few of which work. All W3C standards for storage, thus far, have been supremely unpopular. HTML5 introduces the second iteration of multimedia systems which, despite heavy support from Google and Apple, are highly unlikely to achieve universal usage due to still unresolved functionality and standardization issues.
  7. IndexedDB is touted as a replacement for Web SQL Database but does not contain the same functionality. Web SQL Database is a relational database access system. IndexedDB is an indexed table system. Both are "storage", but are no more related than BerkleyDB and Pilot Analysis Server.
  8. It is not clear what the use case for IndexedDB is, aside from being a last resort way to store data. At no point did anyone apparently say "I'm a web developer. I have to store persistent data for my application" and determine what the developer would want or need.
  9. There's a view that IndexedDB can replace Web SQL Database if third parties create a SQL library in Javascript that uses IndexedDB as the back-end. This is a horrible, horrible, idea and the people proposing it are horrible, horrible people.
  10. Those who promote the notion that someone can always write a SQL library in Javascript would, to my mind, have to concede that there's no reason why it would be impossible to implement SQL within the browser.
  11. Web SQL Database doesn't have to be the only way to implement SQL storage within a browser. If the API is flawed, a replacement that has the same positive features, but few of the negatives, ought to be proposed.
  12. I agree that SQLite's implementation of SQL should not be "the standard". It creates dependencies that should not exist, not just on SQLite, but on the technology stack SQLite is built upon.
  13. SQL is a living standard, despite the best efforts of ANSI. While this is promoted as a disadvantage by Web SQL Database's critics, it's actually an advantage if you're trying to build SQL into a web standard. It means you can define a subset/superset of SQL that benefits from SQL's familiarity but is suited to the task at hand, without having to worry that your subset/superset will be unfamiliar, or that there'll be an argument about your variant not being some arbitrary version that doesn't really fit what you're trying to do.
At the end of the day, I don't necessary disagree with the deprecation of Web SQL Database. It is, however, a good prototype, and its indicative of what developers actually want. So the question I have to ask is why is the W3C even bothering with IndexedDB, let alone actually defining it as a formal standard that every browser will need to support? Is it really that likely that the API will get any serious level of usage?

Would it not make more sense to start over, look at what web developers actually want, and try to put something together they'll use?


  1. Because big players (*cough* microsoft *cough*) don't want people to use non-MS products such as SQLite.

    "This is a horrible, horrible, idea and the people proposing it are horrible, horrible people."
    This is probably the most applicable statement to IndexedDB that I have seen on the internet to date.

  2. @Anonymous - given the proposal originally came from Mozilla, I don't think this has anything to do with Microsoft disliking SQLite - indeed, SQLite is exactly the kind of open source product Microsoft is fond of - compatible with proprietary software, free, helps developers.

    I suspect this, ultimately, boils down to the NoSQL people hijacking the process - about 90% of the "complaints" about Web SQL Database are about SQL ("it's not standardized! It's too complex!") with the only legitimate complaint being against the fact there's only one implementation of the Web SQL API.

    That's my gut feeling anyway - why else would you propose an alternative to Web SQL database that isn't relational, and doesn't even support a subset of SQL?

  3. Paul,

    You get it true. I wanted to implement a small application in a webpage, which should deal with ~1000 items in 6 different structurated tables.

    No way I can do it with IndexedDb. The guys who dropped the Web SQL Database spec should stop writing specs and code for the Web again, just to be conscient where are the needs...

  4. Hi Paul,
    I really would prefer to use IndexedDB for an Android app, but it seems Android doesn't support it as of date of writing.

    The reason for using IndexedDB vs. WebSQL in my opinion is that IDB is faster to write (no need to create transaction functions, etc.) and doesn't really need an ORM. WebSQL basically demands a javascript wrapper of some sort. Add to that that local storage should really be "quaint" compared to powerhouse SQL databases, and a NoSQL-like datastore like IndexedDB makes a lot of sense to me.

    That's my 2c in any case!

  5. IndexedDB is a lot like localstorage to me - not worth much more than a new way of storing cookies. While I agree dependence on WebSQL may not be ideal in it's current form - IndexedDB does not help fill the same role as you said. I am writing a webapp atm that absolutely requires a relational db on the client side for offline functionality - even if this could be done with indexedDB it would likely be ugly and slow... For example I tried to abstract the data to localstorage (because it's the most universal standard) but what they don't tell you is after you have a few thousand entries (still well under the 5MB limit) every browser I tested on every platform slows to a snails pace. key:value storage solutions should only be used for a minimal amount of information at a time, we already have that with localstorage, I don't see how indexedDB offers us anything at all that we don't have already - and it doesn't even come close to fulfilling the needs of developers to use a client side relationalDB.

    My solution for now was to write my own webkit based browsers for each platform I deploy on - this way I can ensure that WebSQLDB is available and that support for it won't be taken away. This wasn't ideal, but for now it was the best solution I could come up with to bypass the ridiculous browser wars.

  6. Actually IndexedDB API is pretty good for both developer and browser vendor. Please try out my library It support both WebSQL and IndexedDB. But the library is thin wrapper for IndexedDB and support full feature of it. WebSQL fail back come close to it.

    IndexedDB have very very good specification (where as WebSQL has none). The end result is browser vender can choose on their favorite implementations. Interestingly, firefox proposed IndexedDB, but internally firefox use Sqlite for IndexedDB API. Chrome has WebSQL, but IndexedDB is implemented in new leveldb

  7. What's needed is for people to write websql code anyway and use workarounds like the horrible, horrible idea to embarrass the browsers who don't support it. I'll be popping up messages like "this runs about 10,000 times faster in anything other than Firefox.