Data Model for Reporting over Windows Azure Tables

One of my colleague tried implemented a browser history mechanism for MVC. Based on this data he would like to generate two simple reports:

1. Top 5 web addresses accessed by a given user per day
2. Top 5 web addresses accessed by all user

When the data store is implemented using SQL Azure, this problem can be resolved very simple.  The question that appeared here is: Can we implement a data store model using Windows Azure Table Services?
I will try to propose a possible data model that is using Windows Azure Table.
In the default implementation (using SQL Azure), there were 3 kind of information that is stored in the SQL tables:

• URL visited
• User
• Date

Because we don’t have an order by, count or a max function in a query over Windows Azure Table we need to think at a model that would help us with this. We will start with the first requirement:

Top 5 web addresses accessed by a given user per day
To be able to solve this problem we need a data model that permit us to refer to URLs for a given day and user.
Theoretically, we can have an unlimited number of tables in Windows Azure Tables (and we don’t need to pay for each table in part). Because of this we can have a different table for each day. In this way the cleanup mechanism will be extremely simple. Also, when we want to access historical, selecting a specific day will be very easily.
We already know that each table from Windows Azure Tables contains two fields that play the role of keys: Partition Key and Row Key. Partition key can be used with success when we have different items types saved in the same table. Entities could be grouped based on the user, because of this we can save in the Partition Key the user id. In this way we will be able to specify a specific date and user.
In the row key we can store the visited URL. Another property (column) will be needed to store how many times the URL was visited in a specific day. On the server we can define a mechanism that will add or increment a visited URL.
The downside of this solution is that we will need to make two different transactions when we want to increment the history counter. One transaction that brings the current counter for the given user and URL (if exist) and another one that update (insert) the counter value.
When we need to generate the TOP 5 for a given day per user we will have to load all URLs for specific user and order this by the counter value. We don’t need to forget that in Windows Azure Tables we don’t have support for Order By and Top N functions of a query – because of this we need to retrieve all the URLs that a user visited in a given day.

Top 5 web addresses accessed by all user

Option 1
To full fit this request we need to create another Windows Azure Table that will store the URL and the counter for each URL. To be able to support Top 5 not only for URL but also for domains we will store in the Partition Key the URL domain and the Row Key the rest of the URL path.
Another solution would be to have in the Partition Key the same values for all rows and in the Row Key the full URL. We don’t want to have in the Partition Key the URL (and in the Row Key the counter) because in this case we risk having a table that is fragmented – when a Windows Azure Table is too big, Windows Azure can split our table based on the Partition Key and move our spited table on different machines (this is not visible for consumer – is an implementation detail).
The biggest downside of this solution in the moment when we need to retrieve the top 5 most visited URLs. We will need to retrieve all the content from our table and calculate the top 5.

Option 2
Another possible solution for this problem is to have more than one table. Based on how many a URL was visited we will be stored in a specific table. For example we would have

• VisitedUrls1to100
• VisitedUrls101to1000
• VisitedUrls1001to10000
• VisitedUrls10001to100000

For example when the URL is in the first table (VisitedUrls1to100) and the counter value will reach 101, that the entity will be moved to the next table and so on. In this implementation we will have a big problem to find a URL to increment the counter. To optimize this we would need to use another table that would store the URL and the table name where our URL counter can be found (VisitedUrls1to100).
The good part of this implementation is in the moment when we need to calculate top 5 and we can retrieve only a part of the URLs.

These were the possible implementation that I see using Windows Azure Tables. I think that this problem is not suitable for Windows Azure Table and a relational database is better for this case.
I didn’t forget about a problem that needs to be solved here – concurrency. I will come with a post tomorrow.
Part 2

Read More

Weak software design - Restrict user access in the web applications

Today I want to talk about a smell that I way sometimes on web application. We will start with a short story to create a context for our smell.
One day a request comes to the developer that he needs to create a page where the user can view a list of items. For each item the user can view details, edit an item and delete it. After a while the manager changes the request: “Only specific users can edit or delete items from the list”. Our developer based on the role of a user identifies what user can edit or delete items and for the rest of the user he decides to hide the two buttons.
When we look over the code something is wrong. If the user knows the URL for the edit page of an item or for deletion, he can edit and delete any items even if is not an admin.
We should never trust inputs that come from users. Every time we should validate on the server side the data from the user and also if he has rights to do execute a command or view a specific page.
In MVC application is very simple to do this. We only need to add an attribute to specific actions that we want to be access based on the user role. Another simple solution is to check if the user is part of a given role using the following method “Page.User.IsInRole(“RoleName”)”.
Be aware when you limit access of a user. For each access to a restricted area you should check the user rights. Don’t trust anything that the user sends to you.

Read More

How to disable the cache content of a web site that is hosted on Windows Azure.

One of my colleagues started to work on a web application that will be hosted on Windows Azure (HTML 5, JavaScript, MVC 4, .NET 4.5). The web application is a simple web application that displays the status of some web services and the load of some servers from a web farm.

The only problem that appear here is that the device from where the application will be accessed. The device has a custom web browser that use Mozilla core. After a will be observed that even if he change JavaScript files, the changes are not visible in the web application, even if he redeployed the application.

The problem is not from Windows Azure, the current configuration of the IIS is not set to cache any content. The problem is from the client browser. By default, Mozilla (Firefox) will try to cache any kind of content.

In the production environment this is the desired behavior, but in the development and testing stage this can be a nightmare and when you don’t have keyword “CTRL+F5” is not so easy to be used.

One solution to this problem is to change the cache configuration of the browser of the testing machines. This can be done very easily if you enter in the URL “about:config” and hit enter. A list of configuration parameters will be listed. The next two parameters need to be set to “false”:

network.http.use-cache = false
browser.cache.offline.enable = false

Another solution is to append to each file from the server that is required by the client a random string (a GUID or timestamp for example). In this way the file will be reloaded from the server each time. The only problem is that we don’t want this behavior in the production environment. A very nice solution for this is to add the session id to each CSS or JavaScript file that is requested from the server:

<script language="javascript" src="js/myFoo.js?id=${Session.SessionID}"></script>

Another solution is to set the expiration date of the content from the configuration file. The only thing that we need to do is to add in the configuration file the following node:

<configuration>
  <system.webServer>
    <staticContent>
      <clientCache cacheControlCustom="public"
      cacheControlMaxAge="00:30:00" cacheControlMode="UseMaxAge" />
    </staticContent>
   </system.webServer>
</configuration>

This configuration can be made also from ISS and from code. If you want to make this confirmation from code you need to add the following lines of code:

Response.Cache.SetExpires(DateTime.Now.AddHours(1));
Response.Cache.SetCacheability(HttpCacheability.Public);

In the moment when you need to set this setting from the code, something is smelly. Maybe you need to use a static content delivery framework like “Mini Static Content Delivery” from CodePlex. Note, this feature is supported in MVC4.
There are a lot of solutions for this problems and a lot of frameworks. In this case the best thing to do was to disable the cache from the client device (for development purpose).

Read More

MVC - What a view should never contain( part 2)

Part 1
Ieri am promis ca revin cu un post despre ce nu ar trebuii sa contina un view.
Pornim de la o clasa PersonModel care are urmatoarea definitie:

public class PersonModel
{
    public string Name { get; set; }
    public int Age { get; set; }
    public string Address { get; set; }
}

Pentru acest model avem urmatorul view:

@model PersonModel

@{
    Layout = null;
}

<!DOCTYPE html>

<html>
<head>
    <title>Person</title>
</head>
<body>
    <fieldset>
        <legend>PersonModel</legend>
   
        <div class="display-label">Name</div>
        <div class="display-field">
            @Html.DisplayFor(model => model.Name)
        </div>
   
        <div class="display-label">Age</div>
        <div class="display-field">
            @Html.DisplayFor(model => model.Age)
        </div>
   
        <div class="display-label">Address</div>
        <div class="display-field">
            @Html.DisplayFor(model => model.Address)
        </div>
    </fieldset>
</body>
</html>

Mai tarziu apare o noua cerinta in aplicatie, ca sa se afiseze si coodonatele GPS a locatiei date. Pentru acest lucru se foloseste o adresa web care stie sa rezolve orice adresa. O implementare simpla este urmatoarea:

@model CIC.PersonModel
@{
    Layout = null;

    string coordLat;
    string coordLong;

    var request = WebRequest.Create(someAddress + "?location=" + Model.Address);
    var webResponse = request.GetResponse();
    using (var contentStream = new StreamReader(webResponse.GetResponseStream()))
    {
        var content = contentStream.ReadToEnd();
        webResponse.Close();
        string[] coords = content.Split(' ');
        coordLong = coords[0];
        coordLat = coords[1];
    }
}
<!DOCTYPE html>
<html>
<head>
    <title>Person</title>
</head>
<body>
    <fieldset>
        <legend>PersonModel</legend>
        <div class="display-label">
            Name</div>
        <div class="display-field">
            @Html.DisplayFor(model => model.Name)
        </div>
        <div class="display-label">
            Age</div>
        <div class="display-field">
            @Html.DisplayFor(model => model.Age)
        </div>
        <div class="display-label">
            Address</div>
        <div class="display-field">
            @Html.DisplayFor(model => model.Address)
        </div>
        <div class="display-label">
            GPS Location</div>
        <div class="display-field">
            @coordLong
            @coordLat
        </div>
    </fieldset>
</body>
</html>

Pagina functioneaza fara nici o problema doar ca in viewul face mult prea multe lucruri. Din el se face un request spre o alta componenta( serviciu), prin intermediul caruia se obtin coordonatele GPS a unei adrese, care se afiseza in view.
Modelul nu contine toate datele necesare pentru a afisa tot ce este necesar. Din aceasta cauza se ajunge sa se faca un apel spre o componenta externa. Nu are importanta daca aceasta componenta este un serviciu extern sau o clasa din assemblyul nostru. Toate datele necesare trebuie sa fie continute de catre model. Orice data trebuie sa ajunga in view prin intermediul modelului.
O solutie este sa adaugam doua propietati in PersonModel care sa reprezinte coordonatele GPS( sau putem sa ne cream o clasa care sa stocheze longitudinea si latitudinea - dar tot ca si o propietate din PersonModel o sa fie).

public class PersonModel
{
    public string Name { get; set; }
    public int Age { get; set; }
    public string Address { get; set; }
    public string Longitude { get; set; }
    public string Latitude { get; set; }
}

Inițializarea coordonatelor urmeaza sa fie mutata in controler. Problema de la care am pornit a fost rezolvata, insa ... apare un smell la orizont. Nu foarte puternic, dar extrem de periculos. Apelul spre serviciu se face direct din controler, cea ce nu e normal. Da, este adevarat ca controlerul ar trebuii sa pregateasca modelul, dar obtinerea coordonatelor ar trebuii sa se faca in alt loc. De exemlu putem sa scoatem acest apel intr-o alta clasa.

Read More

MVC - What a view should never contain( part 1)

Part 2
Suntem intr-o aplicatie MVC. Un model poate sa fie folosit de 1 sau mai multe viewuri.
Intrebarea de unde a pornit totul este Ce ar trebuii sa contina un view?
Daca ne luam dupa carte: Un View trebuie sa contina codul prin intermediul caruia se genereaza interfara grafica asteptata. Viewul se ocupa cu partea de render la date pentru a afisa modelul corespunzator.
In acest caz ce este un Model? Raspunsul scurt si sec pe care il primesc mereu este: o clasa care contine datele ce trebuie afisate in view.
Citisem anul trecut ca un model poate sa fie reprezentat de un web service, de un repository sau de catre un alt bussines layer. Aceasta afirmatie este falsa. Da sursa datelor poate sa fie un web-service, un repository sau un bussines layer, dar datele care o sa fie afisate tot ca o colectie de la 1 la n de de obiecte pe care le primim. Indifierent ca este o colectie de stringuri, o clasa, sau un singur string. Aceste date pot sa fie modelate ca si o clasa.
Daca ne intoarcem la prima afirmatie( modelul este o clasa) - putem sa schimbam raspunsul si sa spunem ca modelul este reprezentat de o colectie de date. In lumea ASP.NET MVC aceasta va fi mereu o clasa( da, chiar si un string este de fapt o clasa in .NET).
Si acuma pot sa pun intrebarea de unde a pornit totul.
Ce ar trebuii sa contina un View?
Doar codul prin intermediul caruia se face render la UI.
Dar daca trebuie sa formatam un element din model, aceast cod unde trebuie sa apara? In controler cand pregateste modelul sau in view. Ati spune ca depinde? Daca formatam un datetime atunci acest lucru s-ar putea face in view sau chiar in template. Daca formatam un obiect mai complex ar trebuii sa fie in controler. In al doilea caz avem o problema cu modelul. Modelul nu este in formatul de care avem nevoie, iar daca incercam sa refolosim un alt model, atunci mai bine ne facem un nou model cu datele de care avem nevoie.
Nu este nimic gresit sa avem cate un model pentru fiecare view. Este gresit sa avem un singur model la 20 de view-uri, iar modelul sa contina date care sa nu fie folosite de toate viewurile sau viewurile sa fie obligat sa faca procesare datelor pentru a putea afisa datele in formatul dorit.
Dar daca avem nevoie sa afisam date din model apeland un web-service. Niciodata sa nu puneti acest apel in view. In unele cazuri poate apelul sa fie facut din Java Script, dar niciodata din view. Genul acesta de logica nu are ce cauta in view.
Totul a pornit in momentul in care am vazut intr-un view un apel spre un web-service facut din C#.
Part 2

Read More

Web application end - how to determine why has stopped

Sa spunem ca avem o aplicatie web care se afla pe IIS. Observam ca aplicatia uneori moare si nu reusimg sa ne dam seama din ce cauza. Din anumite motive "exceptia" care se intampla in aplicatie nu poate sa fie prinsa si aplicația moare fără sa logeze nici o informație. De ce spun "exceptie", deoarece uneori poate sa fie vorba de un pool recycle care sa genereze acest comportament( cauza nu este mereu o exceptie in adevaratul sens al cuvântului).
Ce putem face? O solutie este ca in Global.asax.cs, in metoda Application_End sa obtinem date despre motivul pentru care aplicatie se termina si sa scriem aceste date in trace sau sa le logam undeva. Aceasta metoda o sa fie apelata de fiecare data cand aplicatia este oprita. Din pacate informatiile pentru care aplicatia este oprita se pot obtine doar prin intermedul reflection. Nu avem o alta modalitate sa obtinem aceste date. Daca aplicatia primeste un semnal din exterior ca trebuie sa se opreasca metoda Application_Error nu o sa fie apelata.
Mai jos gasiti un exemplu de implementare a metodei Application_End:

 public void Application_End()
        {
            HttpRuntime runtime = (HttpRuntime)typeof(System.Web.HttpRuntime).InvokeMember("_theRuntime",
                                                                                             BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.GetField,
                                                                                             null, null, null);
            if (runtime == null)
            {
                return;
            }

            string message = (string)runtime.GetType().InvokeMember("_shutDownMessage",
                                                                                 BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.GetField,
                                                                                 null, runtime, null);

            string stackTrace = (string)runtime.GetType().InvokeMember("_shutDownStack",
                                                                           BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.GetField,
                                                                           null, runtime, null);

            Trace.WriteLine(string.Format("Application_End: {0} {1}", message, stackTrace));

        }

Enjoy!

Read More

MVC3 - After RedirectToAction ViewBag is null

Intr-un post anterior am discutat putin despre ViewBag.
Sa vedem ce se intampla daca il folosim in urmatorul mod:

public ActionResult Index()
{
     …
     ViewBag.Error = “some error”;
     return RedirectToAction(“Error”,”Home”);
}

In cazul in care o sa dorim in actiune Error din controlerul Home sa accesam ViewBag.Error, o sa observam ca acesta este null. Informatile din ViewBag sunt pierdute in momentul in care se face un RedirectToAction.
O solutie destul de simpla este sa folosim TempData, care persista intre doua actiune. TempData se poate folosii cu success cand dorim sa trimitem date intre requestul current si urmatorul. Durata de viata a obiectelor din TempData, o sa fie pana la urmatorul redirect (inclusiv). Dupa momentul respective aceste date se pierd. Valoriile din TempData sunt tinute pe sesiune intre actine curenta si urmatorea actiune accesata printr-un redirect.

public ActionResult Index()
{
…
TempData.Error = “some data”;
return RedirectToAction(“Error”,”Home”);
}

Read More

MVC 3 - UrlParameter.Optional

Zilele astea am inceput sa ma uit peste MVC 3 si sa vad ce a adus nou. S-a scris foarte mult, nu am de gand sa desfac firul in patru cu fiecare lucru aparut in MVC 3. In schimb vreau sa va spun peste ce problema am dat.
In MVC 2 cand se declara un map route - unul sau mai multi parametrii pot sa fie optionali. In acest caz cand se face apelul unei actiuni, acesti parametri pot sa lipseasca.
De exemplu putem sa avem urmatoarea mapare:

routes.MapRoute(
    "Drive",
    "drive/go/strada/numar"
    controller = "Drive",
    new
    {
         action = "Go",
         numar = UrlParameter.Optional
         bloc = UrlParameter.Optional
    });

Antetul actiuni ar avea forma:
public ActionResult Go(string strada, int? numar,int? bloc);
Apelul spre aceste acțiuni am dorii sa fie de forma:

http://localhost/drive/go?strada=Dunarii
http://localhost/drive/go/Dunarii/10/5
http://localhost/drive/go/Dunarii

O sa observam ca pentru prima varianta http://localhost/drive/go?strada=Dunarii maparea pe care noi am declarato nu functioneaza. Aceasta problema apare cand avem doi parametrii optionali.
O solutie pentru aceasta problema este sa mai declaram o nou map route pentru cazul in care nici un parametrul optional nu este setat:

routes.MapRoute(
    "Drive",
    "drive/go/strada/numar"
    controller = "Drive",
    new
    {
         action = "Go",
    });

Read More

Cum sa definesti un provider custom pentru cache

.NET 4.0 ne-a adus si posibilitatea de a controla output caching. Pana in momentul acesta puteam sa controlam obiectele la care sa se faca cache. Daca de exemplu avem o actiune la care doream sa facem cache, puteam sa folosim atributul OutputCacheAttribute, prin care puteam defini durata la care sa se faca cache si prin ce parametri acest cache sa difere.

[OutputCache(Duration=20, VaryByParam="none")]
public ActionResult GetItems()
{
    // Executa ceva.
}

In momentul in care se face un call spre actiunea GetItems, se verifica daca aceasta exista in cache si daca cache-ul este valid( nu a expirat iar parametri trasmisi prin VaryByParam sunt identici). Daca conditiile sunt indeplinite atunci rezultatul se incarca automat din cache si se trimite mai departe, altfel se executa actiunea GetItems, iar rezultatul se salveaza in cache.
In cazul in care dorim sa definim un provider propiu pentru output cache este nevoie sa scriem o clasa ce sa mosteneasca din OutputCacheProvider.

    public abstract class OutputCacheProvider : ProviderBase
    {
      public abstract object Get(string key);
      public abstract object Add(string key, object entry, DateTime utcExpiry);
      public abstract void Set(string key, object entry, DateTime utcExpiry);
      public abstract void Remove(string key);
    }

Prin intermediul metodelor care ne sunt puse la dispozitie, putem sa controlam provider-ul in totalitate.
In web.config este este nevoie sa specificam ce fel de output cache sa se foloseasca:

<caching>
  <outputCache defaultProvider="NameOutPutCache">
    <providers>
      <add name="NameOutPutCache"
       type="FullLocationOfOutputCacheImplementation" />
    </providers>
  </outputCache>
</caching>

Pentru a vedea un exemplu de implementare puteti sa va uitati aici: http://galratner.com/blogs/net/archive/2010/06/06/write-your-own-outputcacheprovider.aspx
Mai jos puteti sa gasiti un exemplu de outputcach provider pentru MongoDb:

 public class MongoDbOutputCacheProvider : OutputCacheProvider, IDisposable
    {
        readonly Mongo _mongo;
        readonly IMongoCollection<CacheItem> _cacheItems;
      
        public MongoDbOutputCacheProvider()
        {
            // Initializare coneciune la MongoDb
            _mongo = new Mongo();
            _mongo.Connect();
          
            var store = _mongo.GetDatabase("OutputCacheProviderDB");
            _cacheItems = store.GetCollection<CacheItem>();
        }

        public override object Get(string key)
        {                   
            // Se cauta elementul cu cheia dorita.
            var cacheItem = _cacheItems.FindOne(new { _id = key });

            //Se deserializeaza elementul gasit.
            if (cacheItem != null) {
                if (cacheItem.Expiration.ToUniversalTime() <= DateTime.UtcNow) {
                    _cacheItems.Remove(cacheItem);
                } else {
                        return Deserialize(cacheItem.Item);
                }
            }

            return null;
        }

        public override object Add(string key, object entry, DateTime utcExpiry)
        {
          
            if (utcExpiry == DateTime.MaxValue)
            {
                utcExpiry = DateTime.UtcNow.AddMinutes(5);
            }
          
            // Se adauga elementul in baza de date.
            _cacheItems.Insert(new CacheItem
            {
                Id = key,
                Item = Serialize(entry),
                Expiration = utcExpiry
            });

            return entry;
        }

        public override void Set(string key, object entry, DateTime utcExpiry)
        {        
            var item = _cacheItems.FindOne(new { _id = key });

            if (item != null)
            {
                // In cazul in care elementul deja exista se face update la informatii.
                item.Item = Serialize(entry);
                item.Expiration = utcExpiry;
                _cacheItems.Save(item);
            }
            else
            {
                // Se insereaza un element nou.
                _cacheItems.Insert(new CacheItem
                {
                    Id = key,
                    Item = Serialize(entry),
                    Expiration = utcExpiry
                });
            }
        }

        public override void Remove(string key)
        {
            // Se elimina din MongoDb.
            _cacheItems.Remove(new { _id = key });
        }

        private static byte[] Serialize(object entry)
        {
            var formatter = new BinaryFormatter();
            var stream = new MemoryStream();
            formatter.Serialize(stream, entry);

            return stream.ToArray();
        }

        private static object Deserialize(byte[] serializedEntry)
        {
            var formatter = new BinaryFormatter();
            var stream = new MemoryStream(serializedEntry);

            return formatter.Deserialize(stream);
        }

        public void Dispose()
        {
            _mongo.Disconnect();
            _mongo.Dispose();
        }
}

Implementarea completa se poate gasi aici:
http://archive.msdn.microsoft.com/mag201103OutputCache/Release/ProjectReleases.aspx?ReleaseId=5514

Read More