LinqToSimpleDB Preview

Writing

Update: This project is up over at Codeplex

 

If you follow this blog then you probably know that I am working on a Linq To SimpleDB provider. I finally have it doing something, and so I wanted to post up a preview here in order to potentially get a little feedback.

First of all I wanted to mimic the way that Linq To SQL works, in that I have a context which basically uses a unit of work pattern in order to track changes made to SimpleDB items and allows you to save them back to the database. This context object also allows you to perform other non-query tasks on the database. For instance, if you wanted to try and create a domain, you would do it like this:

  SimpleDBContext.SetAccessKeys("testKey", "testKey");
  SimpleDBContext.CreateDomain("TestDomain");

The call to "SetAccessKeys" sets up a static set of keys that can be used throughout the application. This makes it so that you don't have to pass in your access key and secret key every time. So we are just setting keys and calling "CreateDomain" and that is it. Domain created!

Next we can also do the same thing to delete a domain:

  SimpleDBContext.DeleteDomain("TestDomain");

So, there goes our domain. So, lets pretend that we didn't delete our domain and lets go ahead and populate the domain with a little bit of data. First we are going to need to setup a new context object:

  SimpleDBContext context = SimpleDBContext.GetContext("TestDomain");

After we setup our new context we are going to need to create some new items to put in it. Here is how we can create a new item and put some attributes into it, then add it to the context:

  var item = new LinqToSimpleDB.SimpleDBItem("Item1")
    .WithAttribute("AttributeName1", "value1", "value2")
    .WithAttribute("AttributeName2", "value3", "value4");
  context.AddItem(item);

Then all we have to do is submit the changes back to the database.

  context.SubmitChanges();

This call will go through all of the items that you have added and submit them to the database. It will also remove any items that you may have deleted. This is how you remove an item:

  context.RemoveItem(item);

This will mark the item as deleted so that when you call "SubmitChanges" it will be removed from the database (assuming that it has already been saved to the database) Next, lets look at how to query this data back out of SimpleDB. We just use the context object and we query like this:

  var items = from p in context.Domain where 
    p.Attributes["attrib1"] == "value1" select p;

This will get us our expression tree that we can then execute by calling something like "ToList" or putting it in a "foreach" statment. So here we see us executing our query by running it through a "foreach" statement and checking to make suer that we got our item back.

  foreach (SimpleDBItem dbItem in items)
  {
    if (dbItem.Name == "Item1")
    {
      return true;
    }
  }

I also have the "Union" and "Intersect" parts of the SimpleDB query spec working now, and these just use the built in Union and Intersect operators provided by Linq. A Union would look like this:

  var items = (from p in context.Domain
    where
    p.Attributes["attrib1"] == "value1"
    select p)
    .Union(from p in context.Domain
    where
    p.Attributes["attrib2"] == "value3"
    select p);

I don't have the lazy loading on these queried items done yet (since SimpleDB only returns item names and not the associated attributes), but everything you have seen here is working so far. I am also doing a bit more work with storing numbers in SimpleDB, there is definitely a bit of work that you have to do since Amazon stores everything as text. I am also a bit torn on supporting any kind of query projections, not only is it a pain to do so, but I would have to lazy load every item in order to do a projection, since SimpleDB does not provide a mechanism for pulling back bulk attributes for numerous items.

But for now, I wanted to try and get a little feedback on how this looks so far, and I hope to have an early release out sometime soon.

Loved the article? Hated it? Didn’t even read it?

We’d love to hear from you.

Reach Out

Comments (13)

  1. I have an alpha out that you can try at http://www.codeplex.com/LinqToSimpleDB I really do think that Amazon has a lot of potential with this product, but I have really lost quite a bit my excitement for it while trying to actually use it in a real application. The lack of ordering is simply a deal killer for most applications, especially considering that you are pulling all the data across the net. You can’t honestly expect to sort data client side if you aren’t using EC3. But even if you are using EC3 it is still horribly inefficient.

  2. True. I originally thought the most problematic issue would be the mapping of amazons real world hardware setup to their virtual service setup (SimpleDB, S3). I haven´t tried it myself but it sounded fairly complex to store "bigger objects" in S3 and "smaller objects" in SimpleDB and/or keep pointers to the stuff you keep in S3 in SimpleDB. Working with that thing is probably not fun.

  3. Just out of curiosity… have you thought about the testing story for LinqToSimpleDB?

    IMO, one of the biggest MS made with LinqToSQL was not making it testable. A couple of interfaces and virtual methods would have gone a long to making it easy to mock/test the DataContext and Table objects it returns.

    Just sayin’… it’s something to keep in mind. 🙂

  4. Yes, you are correct in that it is quite difficult to test Linq. The way that I have it in the LinqToSimpleDB provider is that I have a method that I can call on my context to return the text of the query that I am producing. This way I can test my query generation without ever sending it through SimpleDB.

    I also use the Amazon API for connecting to SimpleDB which has built in support for stubbing out calls to SimpleDB. The only problem is that you have to manually supply test responses and SimpleDB is in beta (so it could change a bit), so it really isn’t a great idea for me to put a bunch of fake responses in there when they could change soon.

  5. This is an excellent project. I haven’t had a chance to start playing with your implementation yet. Have you figured out a way to simulate a relational model using SimpleDB?

Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *

More Insights

View All