Updated planning

Just wanted to post the (slightly) more detailed schedule I promised on monday.

I will be doing some writing on my thesis for the next few weeks,
albeit that I will have very limited time due to some exams I need to
study for. My goal is to have about 30 pages (medium quality) by the
first week of next semester (which starts on Feb 14th). I also want to
continue improving the prototype; possibly already adding some of the
ideas I have concerning price discovery.

In order to spread the actual writing work for my thesis, I'm going to
try to write about 5-10 pages each week during next semester (which is
a pretty strict planning, I think). I believe doing this will increase
the quality of the thesis.

I might be some time before I update the blog again because of the
afore mentioned exams.

Evaluation of my personal planning

Being 2 weeks after I made my planning, it is time to evaluate how I have done.
I've been able to write about 16 pages for my actual thesis. These 16 pages address various topics concerning both general market design as specifics about the market I'm designing. I will not put these online yet, since I feel that it needs some additional polishing first.
I've also managed to work a bit on the prototype, but have not been able to add any significant new features.
I did spend quite some time on thinking and brainstorming about price discovery mechanisms for the market design I'm proposing. A concise summary of the ideas originating from this can be found in a small document I made. I'm having a meeting with my promotor about this tomorrow afternoon.

Although that I didn't manage to reach the goals I set up before 2 weeks again, I do think that I did rather well considering the fact that I had some important issues in my private life that needed to be taken care of.

The next few weeks, I won't be having much time since I will need to study for some upcoming exams. However, I'll try to complete some of the things I mentioned in the planning but didn't do yet. I'll also post a updated schedule for the next weeks in one of the coming days.

Making some adjustments to the planning

(I wrote this post a couple of days ago, only now got the opportunity to put it online).
Due to some (unforseen) personal issues, I couldn't work full time on my thesis the past couple of days. Although I'm still going to try reach the goals I had set before; I will be making some minor adjustments to the planning. First of all, I will write some parts of the more introductory chapters of my thesis that answer questions such as:

• what is the problem this thesis tries to solve ?
• what are the (dis)analogies with the electricity market models?
• why different types of markets/auctions exists?

Doing this will give me the feeling that I'm getting more work done, since writing these 'context'-chapters is generally easier than writing about my own market model. The fact that writing the chapter about the market model itself progresses slowly is caused by the fact that the model is not complete yet. That is, although I believe that the current model already has some interesting features and ideas, a few critical points remain to be adressed (primarily; the issue of Price Discovery) properly. Already having done some writing about the model, I think the clarifying these issues first is the way to go. Of course, I'll continue working on the chapter; but I'll take more time to evaluate some of the markets components while writing.

Concerning price discovery, I have already a few ideas; but I haven't been able to address the "variable" priced components (such as outgoing bandwidth, etc) yet. Once I have a more complete picture, I'll create a small powerpoint presentation  and put that online.

On a side note, I have continued working on the (evolutionary) market prototype. At this point, all different components (market, consumers, providers) are actual webservices that communicate through WSDL/SOAP. I also managed to get a single service running on Amazon EC2 (this will come in handy, when I want to demo the prototype), but I haven't tested the whole setup on EC2 yet.

'Till next time :-)

Progress update

A quick post to bring you up to speed.
I've been doing some writing (not too much yet) the past couple of days. I wrote about 5-6 pages about some core aspects of the thesis.
I focussed on Dube's resource sets and the generalization to constraint sets. I also wrote a little bit about the problems of using a CDA (as a mentioned in more detail in my previous post) as primary market model. The text itself is only a rough draft and will need some additional iterations (and of course, a lot of additional content) before I'll post a first version here.

I also did some additional integration work on the prototype. This was not without difficulties however. The main problems lied with passing constraints to webservices (I thought I solved this before, but that turned out to be false). I solved this by separating the properties of the constraint from the the code that validates the constraint (the ConstraintValidator). Turns out this allows for even greater flexibility (since the coupling between constraint and constraint validation is much looser now), which is always a plus.

The next few days, I'll rework the prototype to make use of this new mechanism (I've already made some proof of concept code, but I still need to push it to the whole prototype) and write some more.

Planning the next 2 and a half weeks

As I promised in the previous blogpost: a small overview of what I will be doing in the next 2 to 3 weeks.
By the 18th of January (which is a couple of days before I have an exam, so I need to start studying afterwards), I want to have completed several things for my thesis.

A first rough draft of the chapter of my thesis that will talk about the CRA market design. Writing this will allow me to further refine my thoughts on the design and used mechanism, as well as provide a good 'complete picture'. It might also bring some new unresolved issues to my attention. I estimate that this draft will about 20 pages long.

I also want to elaborate the prototype I've been building. Some of this works includes:

• further integrate webservices (this should be rather simple using Spring + JAX-WS)
• finetune the constraint mechanism (primarily code cleanup)
• further implement the provider-specific matching behavior (this may take some time)
• add matches to billboard (price discovery)
• general code cleanup, refactoring, improvements
• very simple user management system (this is more a personal interest to have a look at the Spring Security framework).

I also hope to elaborate a bit on possible price discovery mechanisms for brokers. The goal is to have a small powerpoint presentation with some new ideas.

I know this is a rather ambituous schedule (especially if you know that I also need to take care of some things for other classes), but after 2-3 weeks of working on other projects, I really feel like getting started again :-)

'till one of the next days :-)

My thesis: an update

I know I haven't posted anything in a while here. This has a lot to do with the way I tend to write blogposts. Most of the time they become a lot longer than I first anticipate (such as now). Most of the time, I tend to have many ideas; and writing all of those down takes a lot of time.
I do believe however, that I should revive this blog since blogging is a great way to communicate latest findings fast. When keeping it up to date regularly, it also provides a good overview of the progression that is made. Additionally, my thesis does focus more on the 'ideas part' than on the 'implementation part'. Obviously, writing those ideas down is a good way to reflect on them.

Thus, my new year resolution is to update this blog far more frequently with smaller, but informative blogposts.
Since I haven't posted anything for about 2 months, I thought it was a good idea to create a small overview of the things I have been doing up until now (although nothing appeared on this blog, I have of course continued working on my thesis).

Chronological overview of the past months



A short update on where we are today

As you can see in the overview, I mention the term CRA a few times. The idea of the using the Continous Double Auction as the market mechanism has been somewhat altered to what I like to call a 'Continuous Reverse Auction' (or CRA for short). The idea behind the CRA is to use a Reverse Auction (which relates to some of the ideas I already discussed in a previous blogpost (http://thesisjorisroovers.blogspot.com/2010/10/component-based-offers-and-custom.html) in a continuous way. Rather than having several bidding phases for a single bid (such as in a Reverse Auction), providers will now only provide a single price for a single bid (when a bid comes in, the market contacts all of the providers to ask for a price). A bid is matched against the cheapest offer for that bid.  The price lowering effect that classic Reverse Auction tends to result in, is achieved by having a lot of bids (and matches) at after each other. By publishing anonymized matches to a kind of public billboard, providers will have the tendency to lower their prices in order to 'win' more matches. This means that in time providers will tend to show incentive compatible behavior.
The main benefits of the CRA are that it is continuous (matches happen in real-time), and allows great matching flexibility. The limits of what this model can match is defined by the limits of the used bidding language (in our case, the used constraint mechanism), which is a nice feature to have.
However, when working with the Continuous Reverse Auctions, consumers are price takers (compared to the CDA, when they also influence the price of the match). A disadvantage of this is that maximal market efficiency is not reached. That is, it might be that there are consumers that are willing to pay more for a match than the CRA will ask since they attach more value to it. Although this benefits the consumer in this case, the CRA market clearly is not able to provide an optimal clearing here.

However, in the current market situation; it is difficult to come up with a mechanism that does provider maximum effiency. This is primarily caused by the fact that currently cloud providers claim to provide 'unlimited' capacity, which suggest that resources are never scarce and that fluctuations in demand do not affect the value the provider attaches to its remaining (unused) capacity. This is obviously not true (this idea goes against every demand-supply based market principle); but at the moment, this is what providers try to sell consumers. As long as providers keep using this principle to price their offers (that is, an offer is always priced the same, independently of demand. Additionally consumers that are prepared to pay more than other consumers are not treated differently), maximum market efficiency is not a achievable goal. Note that although Amazon's spot instances do have hourly fluctuating prices according to demand, it does not have admission preferences among consumers.
However, you can ask yourself whether maximum market efficiency is a desirable property in all cases. I believe that in a Business-to-Consumer environment, this is not neccesarily the case. An end-user is probably less interested in always providing a price when wanting to use a utility (even if advanced price discovery tools are available). Bidding/Procurement is more the task of a broker (as is the case in e.g. the electricity market), who later on uses seperate mechanisms (average price, flat fee, day/night tariffs, ...) to bill its clients.

I believe that the CRA model contains some good ideas in the case where the consumers are price takers (as is currently the case in the cloud ecosystem). When consumers also need to influence the prices, we need more a hybrid between CDA and CRA in which brokers take the roll of consumers. In such model, the brokers no longer simple take the price of the cheapest provider; but instead they will submit bids. The bid of the broker will be matched against the offer of the provider that made the cheapest offer. By doing this, we achieve maximum market efficiency for that single bid. When wanting to achieve maximum efficieny among multiple bids, the CRA model (or any other continuous model, such as the CDA, for that matter) can no longer be used since multiple bids then need to be compared with each other. This makes the clearing market a lot more complex. Additionality, since it is impossible in a real-time ecosystem (such as a cloud market) to know whether a better bid or offer will arrive or not, such systems will always need to use heuristics to approximate maximum market efficiency.
The CRA model still needs some more elaboration, and some questions remain to be answered (price discovery stays a big issue), but I think that in terms of flexibility it already passed a first test.

I will continue to work out some of the details in the coming weeks.

So, with this medium-sized post, I updated the blog with the most salient ideas behind the CRA model. More detailed information can be found in the pointers provided by the overview document embedded above.

I'll be writing a post about my plans for the coming weeks later today.