Microservices and the Database Collective

The Prime Directive

So, you've taken the plunge and converted your monolith into discrete, single-purpose microservices. You've established a set of boundaries and service ownership that extends to the databases, so each microservice has its own little database. You've even instituted the CQRS model to separate your write operations from your read operations, and only the service that owns the database can actually make changes to it; all other services are afforded read-only permissions through a carefully constructed query library that only exposes what needs to be exposed. So now, you can breathe a sigh of relief: the days of the tightly-coupled components are over. You can do whatever you want with Service A with abandon, knowing that you'll never break Service B in the process.

Unless you change the database schema for Service A. Because, you see, that carefully constructed query library that Service A owns is a critical dependency for Service B. Service B can happily query the Service A database without caring anything about the schema--until it changes. Then that query library explodes like a furious kitten and takes down your services with it.

No problem, you say. That's an easy fix! Just update the query library in Service B and redeploy!

Do you see the problem? If Service B must be updated because of a change to Service A, then you have just violated the microservices prime directive: changes made to one service must never interfere with another service.

We are Database. You will be assimilated. Resistance is futile.

All development shops have their version of The Database. This is where business critical information is stored, on which not just one but multiple components of the software rely. You may have multiple databases already, but you know which one is The Database: it's the one that causes all systems to halt when something breaks or, worse, changes.

It's true that breaking up The Database into smaller, bounded databases relieves some of the pain, but no matter what you do, you're still going to have your mission critical data somewhere, and it's very likely that almost every microservice in your architecture is going to need to query that data at some point in one way or another.

When Martin Fowler and Greg Young describe CQRS, they speak of the concept of separating the responsibility of write operations from read operations. The core benefit of CQRS for them is the ability to scale the write operations independently from the read operations. Thus, although they mention having separate data models and possibly separate databases for the read and write operations, it's not fundamental to the pattern to have the data stored separately.

When Udi Dahan describes CQRS, he does so in the specific context of service boundaries. He discusses the speed and convenience of having your data stored separately in exactly the format that your consuming service needs it to be in, eliminating the need to transform it. 

The Data Neutral Zone

In Udi's blog post about CQRS, he is diplomatic about where you should put this read-only store. It could be another table in the master database; it could be an in-memory cache; it could be a document database; it could even be a stored procedure or view or code-based transformation. This last option is essentially what your query library is probably doing for you, which is great, but we've already identified that it is fragile if the master database schema changes. Thus, I will assert that a view, stored procedure, or code map is not a viable solution. By having the read-only data stored separately from the master schema, you can altogether avoid breaking the services that rely on it when you need to update the master schema.

You might be tempted to simply create versioned views or versioned stored procedures in the master database and call that the solution. And it very well might be a decent solution; after all, you're preserving the format of the read-only data for the services that are not yet updated, and you're providing the new format for anything that cares to know about it. One thing haunts me about this solution, however, and, in fact, it haunts me about the query library, as well: who owns that format? Which service does it belong to? Sure, the data belongs to the service that owns the database, but if that service never uses that query format, then does it really own it? Or does the consuming service own it? If the answer is the latter (which I think it is), then what business does that view or stored procedure have in the master database to begin with, let alone several versions of it? I argue that the consuming service should be responsible for determining its own query formats because it knows what it needs, and it shouldn't have to work around the constraints of the master database to get it.

You might still want to keep the query store in a central place because more than one service queries the data in the same way. At least, for now. That might change in the future; Service C might need a slightly different format. No big deal, just create a new version of the store. Now who owns it? It gets more complicated when you start thinking about what to name these query stores. If you simply name it based on the data or format, you'll quickly lose track of which services rely on which store, making it more and more difficult to maintain and later prune. And your highly-tuned sense of service boundaries should be highly offended by the idea of naming the data store to reflect the service(s) that use it. And anyway, you should know by now that reuse is a fallacy, even when it comes to your database. There's really no such thing as a data neutral zone--every service is eventually going to need something different, and there will be conflicts.

There Are Four Databases

You've already agreed at the beginning of this journey that each service should have its own database that it is responsible for updating. The next logical step is to let each service have its own database that it is responsible for querying. It can already freely format and query the data that it owns. Now, let's give it the power to freely format and query the data that it borrows from the other services.

I would hope that you're already using an event-driven or message based architecture with your microservices, so you're already familiar with the concept of eventual consistency and stale data. Indeed, if your commands are separate from your queries, and you query the data before a command is processed, you'll get stale data. You're used to it, your users are used to it (or, even better, protected from it), and your software is used to it. So your service won't mind if the data it queries from its own copy of the data is a little out of date. It will get there eventually. In fact, you have the power to decide whether it really even matters if the data is up-to-the-minute, up-to-the-hour, or otherwise. In some of Google's data-hungry applications, you don't get accurate and complete data for 24 hours. Yes, you'll have to write or implement some mechanism to copy the data over, but it might be easier than you think. You're probably already publishing events after commands are successfully written. Have your consuming services listen for those and save whatever data the events contain. Or use a scheduled job to copy the data over in bulk.

But, you say, we have exact copies of the same data in more than one place! I'll quote Udi here: "So?" In all likelihood, it won't be an exact copy, since you'll have taken the time to store it in exactly the format your consuming service needs. Whether this means it is flattened or aggregated, it probably looks a lot different from what you started with. And even if it is exactly the same, what does it matter? Storage is cheap, especially when compared to the alternative: the cost of your time unwinding the mess you made with your spaghetti services. And remember that you don't need to use a relational database to store this data; it can be a document database, an in-memory cache, or whatever works for you--as long as the consuming service retains ownership.

This method of creating data stores that belong to the services that query them nets you a few things. First, you've truly eliminated the dependency on The Database, and you'll never have to redeploy another service because you've changed the schema for another. Second, you've given control to the consuming service to transform and manipulate the data however it needs to without worrying about how it might affect other services, and your developers know exactly which service is using that format without combing through every repository to find a reference to it. Third, you've lightened the load on The Database. Less contention is always a good thing!   So, stand and proudly proclaim, "There are four databases!" Or five. Or six.

What I Use - Visual Studio Extensions

This post is mostly so I can reference which extensions I regularly use. It can be a bit of a pain to try and remember the ones I don't use very often when I switch between my work and home PCs. It would be awesome if VS could sync my extensions and their settings. Maybe in VNext.

Just a quick note: I do not work for any of these companies. Where the extensions are not free I paid for them with my own money.

Must Have (every day use)


The first productivity tool I used for Visual Studio (way back for VS 2005). Yes, I've tried ReSharper. In fact, there was a time where I was using both CodeRush/Refactor! and ReSharper via some hackery. I've looked at other tools, but always come back. Not free, but worth the money considering you only have so many keystrokes before you die

I also use the CompilationResult plugin to add a little humor to my life. Green encouragement messages when the code compiles, Red humiliation messages when it doesn't. Always fun when presenting.

Bonus info: DXCore is a free plugin framework and the Compliation Result plugin works with it.

Web Essentials

Everything Mads does is magic. Web Essentials is a playground where the Aspnet team tries out new functionality that may eventually be baked into Visual Studio. This is the second extension I install.

Tabs Studio

Before the Productivity Power Tools and Visual Studio added much needed enhancements to the tab well in Visual Studio there was Tabs Studio. I originally installed it to get better control over tabs in SQL Server Management Studio, but quickly began customizing the rules and options for VS (Source Control Explorer tab always left most ftw!). Not free, but reasonably priced considering it works will basically all versions of VS and SSMS.


Errors in red in the output window, yes please! Colorizing the output window is awesome, and should be baked in to VS. If you've ever tried to troubleshoot a binding issue in WPF you know how difficult it is to spot in the output window.

Productivity Power Tools

Lots of features, some make it into the next version of VS some don't. Updated fairly regularly with additional tweaks. My favorites: syntactic line compression, peek help/definition, and timestamp margin. Make sure to disable the custom document well if you're using Tabs Studio as they conflict with each other. 

TFS Power Tools

If you administer TFS, these tools make your job a lot easier.


Add web.config style transforms to any config file. I don't understand why this isn't baked into VS already. Unfortunately, this is the last version of the add-in unless something changes - see Sayed's post for details.


Lets you F12 directly into .Net Framework source! No more annoying [From Metadata] tabs that only show you signatures.

Error Watcher

Another one from Mads, shows you errors and warnings as an editor overlay for the file you're working on. No more digging into the Error window.

File Nesting

Want to your .min file nested under the raw js/css? How about concrete implementation files under the interface? This guy lets you nest files in solution explorer like a boss! Has a bunch of rules out-of-the-box that handle pretty much all the situations you can think of. Mads is a machine!


Want nicer /// xml doc comments? Want the comments filled out based on the names of your classes/methods/properties? Does your company have a specific format for comments? This does it all and more. Free version handles all the basics. Paid version has additional features. I use the free version, but thinking about shelling some cash out since I use it all the time.


Do you use T4MVC in your ASP.Net MVC apps? If you do, this automates the right-click > run custom tool dance you always forget to do after saving changes to a controller or static resource. 


Just recently installed this and I like it already. Gives you little "encouragement" messages when you save files. You can also add some skepticism messages to keep it lively.

Microsoft CodeLens Code Health Indicator

If you're rockin' VS 2013 Ultimate you know how awesome CodeLens is. This adds an additional indicator displaying a simple color to represent the maintainability of the code by utilizing various metrics (e.g., cyclomatic complexity, line count, etc.). I find that if this guys shows red, I take a second look to see if I can refactor for cleaner code.


Adds a "My History" item to Team Explorer that will list out what you've been doing in TFS. Can't remember the ID's of the work items you were just working on, it will tell you without the need to run a query or go to your sprint board.


Ever created a new class, write a bunch of code, and delete it only to realize you need it after all? This little guy keeps track of everything you do locally and independent of source control making it best suited for the times you're slinging code and not ready to check-in yet. It works well for this, but it's also interesting to see how you work throughout the day.

Less Used but still essential


Ever wanted to be able to Add New Item > nuspec? This guy adds all kinds of new file template goodness. A lot templates here and all very useful for getting started with new technologies like Angular.

Image Optimizer

I don't always need to optimize images, but when I do, I use Image Optimizer. Right-click > optimize on single files or an entire folder. No surprise here, another one from Mads.

Pretty Paste

Have you ever been frustrated when pasting code from a blog into VS only to have the line numbers and weird white spacing come with it? Mads was, so he wrote this extension will fix all of that for you, automatically.

Recently added to my list

Add Empty File

Does this sound familiar: you need to create a file, but VS doesn't have a template for it so you create it in the file system, show hidden items in solution explorer, and then hit include in project? This simple extension will let you create that empty file right in solution explorer without the hassle. That's right, Mads created this one too.

Bing Developer Assistant

This one's interesting. Shows you code samples, while coding, in the intellisense popup. I simultaneously like this and get annoyed by it, but not so much that I uninstalled it. Worth checking out if you're constantly googling (with Bing!) to find out how to use some API.

Cobisi Routing Assistant

Powerful routing tools. See all of your routes and try out changes to routes. I don't do much with custom routing, but when I have issues this comes in handy.

Knockout Generator

Generates Knockout view models from C# classes. Very cool.

TFS Administrators Toolkit

Recently installed this one. Has some nice features for managing subscriptions and pushing updated work item type definitions to multiple team projects. if you manage a customized TFS project template, you will find this automates some of those tasks that you had to script.

NuGet Reference Switcher

Let's you easily switch between NuGet references and project references during development of NuGet packages. Just installed this one, looking forward to leveraging it soon.

In Conclusion

Which extensions do you find invaluable for day-to-day development?

Enum Support in MVC 5.1

MVC 5.1 introduced some really great support for enums as announced in the MVC 5.1 release notes.

Given the following enumeration and model class...

public enum Heroes 

public class EnumSample
     public Heroes Hero { get; set; }

...using the following built in HTML helper method...

@Html.EnumDropDownListFor(m => m.Hero)

...renders a drop down with the enumeration names:





Cool! Exactly what we expected: a drop down list with the enumeration options. What if we wanted to show the enumeration names in a more friendly manner, like putting a space in WonderWoman?

Turns out, this is no problem for the new helper. The text that is displayed honors any DisplayAttributes that have been applied to the enum values.

public enum Heroes
     [Display(Name="Wonder Woman")]

public class EnumSample
    public Heroes Hero { get; set; }

Line 5 above is where the magic happens. Notice that I did not use a DisplayNameAttribute as you might have expected. The reason is that the DisplayNameAttribute is not valid for Fields and will throw a compiler error.

@Html.EnumDropDownListFor(m => m.Hero)

now renders the following:





Sweet! Now we have nice, friendly display values.

Where it breaks down

You would think that the added support for creating drop down lists and honoring display attributes would also apply to simply displaying a enumeration value

@Html.DisplayFor(m => m.Hero)

and… you would be wrong.


The solution

There are a few solutions available if you google it (with bing, of course!). Unfortunately, most of them are related to creating a drop down list for enum values. There are a couple of answers on stackoverflow that provide simple utility classes/methods to wrap up the reflection required to get the name from the attribute.

I was really looking for something cleaner and easily discoverable. So, I created my own HtmlHelper specifically for displaying the names of enum values.

@Html.EnumDisplayFor(m => m.Hero)

Let's take a look at the code:

public static MvcHtmlString EnumDisplayFor<TModel, TValue>(this HtmlHelper html, Expression<Func<TModel, TValue>> expression)
    return EnumDisplayNameFor(html, expression);

public static MvcHtmlString EnumDisplayNameFor<TModel, TValue>(this HtmlHelper html, Expression<Func<TModel, TValue>> expression)
    if (html.ViewData == null || html.ViewData.Model == null)
        return MvcHtmlString.Empty;

    var metaData = ModelMetadata.FromLambdaExpression(expression, html.ViewData);

    if (EnumHelper.IsValidForEnumHelper(metaData))
        var enumType = metaData.ModelType;
        var enumName = Enum.GetName(enumType, metaData.Model);

        var enumField = enumType.GetField(enumName, BindingFlags.Static | BindingFlags.GetField | BindingFlags.Public);

        if (enumField != null)
            var attr = enumField.GetCustomAttribute(inherit: false);

            if (attr != null)
                string name = attr.GetName();

                if (!string.IsNullOrWhiteSpace(name))
                    return MvcHtmlString.Create(name);

    return DisplayExtensions.DisplayFor(html, expression);

Basically, all we're doing is checking to see if the enum value in the model has a DisplayAttribute, and if so, try to grab the Name from it.

The EnumHelper.IsValidForEnumHelper method is part of the framework, and I used it here to make sure that the model value is an enum and that it does not contain flags. I'm not sure why having flags would cause an issue, but I included the check to maintain consistency with the EnumDropDownListFor method.

The DisplayExtensions.DisplayFor method was used here for consistency with the framework as well.

What about discoverability?

Adding the namespace to the web.config file did not make these extension methods available in my views. I'm not sure why. The only way I could get them to be available was to add the explicit using at the top of each view where I needed to use them. Needless to say, not very good for discoverability.

After fighting with this for some time, I ultimately decided to put these in the System.Web.Mvc.Html namespace. As dirty as it feels to do that, the nice side effect is that by simply referencing the DLL, these methods instantly become available to all of your views without having to muck with web.config or add additional namespaces to your views. The ultimate in discoverability!

In conclusion

I'm pretty satisfied with the solution that I've come up with. These extension methods are available on NuGet:

Install-Package RedWall.MVCSugar

The source, including a sample project, is available on GitHub.

No, I Don't Want a Ticket for the Women's Luncheon

Women in Technology

Perhaps one of my greatest pet peeves is the term "women in technology." The hatred of this term began for me in middle school when the school's IT director set up an elective class titled "Women in Technology." She came to me explicitly and asked me to join because I had just participated in creating the school's first website (and if you're out there, Mrs. Weber, thank you for jumpstarting my web career). I eagerly agreed that it was a perfect fit. She meant well, I'm sure, but the only memory I have of the class is a group of teenage girls, myself included, at the Rocky Mountain Chocolate Factory putting tiny sweaters on tiny stuffed bears, sipping hot cocoa, and talking about boys. What did this field trip have to do with technology, let alone women in technology?

So when I attended DevConnections in Las Vegas for the first time, it was with great hesitation that I accepted the ticket for the Women in Technology Luncheon. Meet us for lunch Wednesday afternoon, and celebrate women in technology. By Wednesday morning, my husband--also attending the conference--was cringing every time I noticed the ticket tucked in my badge holder and bracing himself for another tirade.

My initial hesitation about going to the luncheon had gradually grown into a vehement denial that the luncheon should exist in the first place. What was the point of it? Was it a support group? A private club with a "no boys allowed" sign stuck on the table? I had come to this conference to learn about exciting changes in the industry, and here I was, 15 years later, being invited to clothe teddy bears again. In reality, the luncheon was probably a pleasant gathering of intelligent women--I don't know, I couldn't bring myself to go. But to me, it was more than a luncheon: it was a giant neon sign that flashed, "Look at us. We're different."

Pride in Successful Women

Don't get me wrong. I am not ashamed of being female, and I don't agree that women should be excluded from or marginalized in an industry that is, currently, male-dominated. I admit to feeling great pride when I see successful women in the industry, such as Michele Leroux Bustamante and Julie Lerman. In fact, I loved introducing my team of developers (all male) to WCF and Entity Framework, not only because the technologies are fantastic, but also because I could point to the names on the covers of the books and say, "See? You don't need a beard or an extended forehead to be an expert."

What bothers me about "women in technology" is the apparent need to beat their bosoms and proclaim that the industry must change to accommodate them, must recognize that they exist, must treat them equally!! -- but differently.

Feminism is a Humanism

In college, I grudgingly took a class entitled "Feminist Bioethics." Great, I thought when I signed up for the class, an entire quarter on abortion. Still, it was the only bioethics class at the university, so I decided to take a chance.

I was surprised by the class, not only because of the variety of bioethical issues that were specific to women that had never occurred to me, but also because it taught me what feminism actually is--and what it is not.

Feminism is not the idea that women are better than men. Feminism is not the idea that women should be vaunted and revered. Feminism is not the idea that men should pay for millennia of misogyny and inequality. Feminism is simply the idea that, yes, men and women are different--and it doesn't matter. When it comes down to it, both men and women are human. This transcends gender, sex, roles--it is a simple and undeniable truth. The core belief of feminism--more aptly named humanism--is that anyone, male or female or otherwise, is entitled to the basic rights and privileges and respect that is due to any human.

That's it! Gender plays no part in your being human. I am proposing that gender need not play any part in your career in technology, either. As Stacey Mulcahy says, "I hope that if someone chooses to toss an adjective in front of your job title, that it is about your qualifications - not your gender." By calling yourself a "woman in technology" and hosting a special luncheon, you are putting that adjective on your job title yourself.

"I totally fucking disagree!"

Which is why it is infuriating when women in technology start imposing themselves on the industry and demand that it change to accommodate them. It is that attitude that leads Uncle Bob to admonish male developers that there are ladies present and to act accordingly. It's not Uncle Bob's fault; his heart is in the right place. He admits that he doesn't understand what the hell women in technology want. That’s because women in technology want it all: praise for standing up against adversity and no adversity. At the same time. Of the examples that Uncle Bob brings up in his article, only one woman seemed to have any sense: the one that yelled, "I totally fucking disagree."


Yes, there are some challenges with being a woman in technology. But how many of those are self-imposed or imagined? I kept my pregnancy a secret from my developers for months because I didn't want to draw attention to the fact that I am female. I didn’t want to lose any of the respect that I had worked hard for as their peer and later as their manager. Did any one of them care when I finally revealed it? Did it affect their opinions of me? Of course not. It was a ridiculous fear, just as ridiculous as the idea that men making sex jokes at a conference threatens women's place in the industry.


Uncle Bob makes the point that those types of jokes are unprofessional no matter who is in the room, and I do agree with that. But that's an entirely different argument altogether. It has nothing to do with women. Generic sex jokes don't hurt women any more than they hurt men. But what if the joke is directed at someone in particular?

Two of my male colleagues joked the other day that a female colleague had to have been so successful because she gave sexual favors to an industry leader, not because she had any real skill. My temper flared. This is exactly what women have to put up with! But as I thought about it later, I realized that, had they been talking about a man, I probably wouldn't have cared. But wouldn't the joke have been just as hurtful? What if they had just said that she was a brownnoser? Isn't that essentially the same thing? And the bottom line is that I also have no idea why this particular person is so successful--she doesn't appear to have the skills to match her success. For all I know, my male colleagues could have been on the right track.

And if we want to talk about professionalism, why should men have to put up with women bursting into tears at the slightest conflict? "Not all women are like that! That’s a stereotype!" Oh, but all brogrammers are misogynists?

Be proud of who you are because of what you do

Meanwhile, the most successful women in technology are respected because they are experts in their craft, not because they are loud and disruptive about issues that, at the end of the day, aren't holding any of us back in our careers. If you want a raise, then ask. If you want a promotion, then ask. If you see actual evidence of discrimination, then it is your duty to make it known. The statistics about women being underpaid and underemployed in comparison to their male counterparts are real. Those statistics have to change.

But keep in mind that intentionally labeling and setting yourself apart is directly sabotaging your goal of desensitizing the industry to the novelty of women. The most important part of who you are in your career is not your gender, so stop categorizing yourself as "women in technology." That label does nothing for your salary, it does nothing for your job title, and it does nothing to prove that you are an equal. Only your work and your passion can do that.

The beginning of something awesome

After years of false starts, we have finally started a blog. The idea is for all of us at Red Wall to contribute and keep track of the things we are doing, or going through, on a regular basis.

We will blog on a broad range of topics that we think may be valuable for world at large. or thoughts that we don’t want to lose track of.

Our intention is to provide value to the community and we invite you to take this journey with us.