This post was migrated from Justin’s personal blog, 'Codethinked.com.' Views, opinions, and colorful expressions should be taken in context, and do not necessarily represent those of Simple Thread (and were written under the influence of dangerous levels of caffeination).

Update: This project is up over at Codeplex

 

If you follow this blog then you probably know that I am working on a Linq To SimpleDB provider. I finally have it doing something, and so I wanted to post up a preview here in order to potentially get a little feedback.

First of all I wanted to mimic the way that Linq To SQL works, in that I have a context which basically uses a unit of work pattern in order to track changes made to SimpleDB items and allows you to save them back to the database. This context object also allows you to perform other non-query tasks on the database. For instance, if you wanted to try and create a domain, you would do it like this:

  SimpleDBContext.SetAccessKeys("testKey", "testKey");
  SimpleDBContext.CreateDomain("TestDomain");

The call to "SetAccessKeys" sets up a static set of keys that can be used throughout the application. This makes it so that you don't have to pass in your access key and secret key every time. So we are just setting keys and calling "CreateDomain" and that is it. Domain created!

Next we can also do the same thing to delete a domain:

  SimpleDBContext.DeleteDomain("TestDomain");

So, there goes our domain. So, lets pretend that we didn't delete our domain and lets go ahead and populate the domain with a little bit of data. First we are going to need to setup a new context object:

  SimpleDBContext context = SimpleDBContext.GetContext("TestDomain");

After we setup our new context we are going to need to create some new items to put in it. Here is how we can create a new item and put some attributes into it, then add it to the context:

  var item = new LinqToSimpleDB.SimpleDBItem("Item1")
    .WithAttribute("AttributeName1", "value1", "value2")
    .WithAttribute("AttributeName2", "value3", "value4");
  context.AddItem(item);

Then all we have to do is submit the changes back to the database.

  context.SubmitChanges();

This call will go through all of the items that you have added and submit them to the database. It will also remove any items that you may have deleted. This is how you remove an item:

  context.RemoveItem(item);

This will mark the item as deleted so that when you call "SubmitChanges" it will be removed from the database (assuming that it has already been saved to the database) Next, lets look at how to query this data back out of SimpleDB. We just use the context object and we query like this:

  var items = from p in context.Domain where 
    p.Attributes["attrib1"] == "value1" select p;

This will get us our expression tree that we can then execute by calling something like "ToList" or putting it in a "foreach" statment. So here we see us executing our query by running it through a "foreach" statement and checking to make suer that we got our item back.

  foreach (SimpleDBItem dbItem in items)
  {
    if (dbItem.Name == "Item1")
    {
      return true;
    }
  }

I also have the "Union" and "Intersect" parts of the SimpleDB query spec working now, and these just use the built in Union and Intersect operators provided by Linq. A Union would look like this:

  var items = (from p in context.Domain
    where
    p.Attributes["attrib1"] == "value1"
    select p)
    .Union(from p in context.Domain
    where
    p.Attributes["attrib2"] == "value3"
    select p);

I don't have the lazy loading on these queried items done yet (since SimpleDB only returns item names and not the associated attributes), but everything you have seen here is working so far. I am also doing a bit more work with storing numbers in SimpleDB, there is definitely a bit of work that you have to do since Amazon stores everything as text. I am also a bit torn on supporting any kind of query projections, not only is it a pain to do so, but I would have to lazy load every item in order to do a projection, since SimpleDB does not provide a mechanism for pulling back bulk attributes for numerous items.

But for now, I wanted to try and get a little feedback on how this looks so far, and I hope to have an early release out sometime soon.

13 Comments

Justin Etheredge

I have an alpha out that you can try at http://www.codeplex.com/LinqToSimpleDB I really do think that Amazon has a lot of potential with this product, but I have really lost quite a bit my excitement for it while trying to actually use it in a real application. The lack of ordering is simply a deal killer for most applications, especially considering that you are pulling all the data across the net. You can’t honestly expect to sort data client side if you aren’t using EC3. But even if you are using EC3 it is still horribly inefficient.

Reply
Elmar

True. I originally thought the most problematic issue would be the mapping of amazons real world hardware setup to their virtual service setup (SimpleDB, S3). I haven´t tried it myself but it sounded fairly complex to store "bigger objects" in S3 and "smaller objects" in SimpleDB and/or keep pointers to the stuff you keep in S3 in SimpleDB. Working with that thing is probably not fun.

Reply
Steven Harman

Just out of curiosity… have you thought about the testing story for LinqToSimpleDB?

IMO, one of the biggest MS made with LinqToSQL was not making it testable. A couple of interfaces and virtual methods would have gone a long to making it easy to mock/test the DataContext and Table objects it returns.

Just sayin’… it’s something to keep in mind. 🙂

Reply
Justin Etheredge

Yes, you are correct in that it is quite difficult to test Linq. The way that I have it in the LinqToSimpleDB provider is that I have a method that I can call on my context to return the text of the query that I am producing. This way I can test my query generation without ever sending it through SimpleDB.

I also use the Amazon API for connecting to SimpleDB which has built in support for stubbing out calls to SimpleDB. The only problem is that you have to manually supply test responses and SimpleDB is in beta (so it could change a bit), so it really isn’t a great idea for me to put a bunch of fake responses in there when they could change soon.

Reply
Justin Etheredge

Hmm, now that SimpleDB has sorting, it would be a much more interesting option. I have a project coming up which might give me some time to work on this.

Reply
Jose Basilio

This is an excellent project. I haven’t had a chance to start playing with your implementation yet. Have you figured out a way to simulate a relational model using SimpleDB?

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *