Inphina Thoughts

We have been working on Google App Engine since its early days and have developed and ported several applications on the platform for our clients. One of the major hinderance for our Enterprise clients for not being able to move to GAE has been Datastore being the only storage option available till date. Major reasons for these enterprises not using Datastore are:

• (In case of migration) Substantial efforts in adapting the application code base to Google Datastore specifications
• Potentially huge data-migration efforts
• Vendor Lock-In as both the application code and data residing in Datastore can’t be taken to an in-premise Infrastructure or another cloud platform seamlessly

Keeping all these requests and potential business opportunities in mind, Google has recently announced Cloud SQL.

What is Google Cloud SQL

• It’s a Relational Database on Cloud offering from Google.
• Offers MySQL Database environment with JDBC support for Java applications and DB-API support for Python applications.
• Being a service on Cloud, it offers traditional advantages of any Cloud service like no maintenance, administration, replication and backup efforts
• To ensure high-availability, data gets replicated across multiple geographic regions
• Easy to use Web-Interface for managing multiple Instances

Google Cloud SQL & Google App Engine

A very good integration has been provided for your applications running on Google App Engine to use Google Cloud SQL as back-end storage mechanism. Furthermore, one SQL instance can be shared across multiple Google App Engine applications. This feature should be quite helpful as often we have a family of Web Applications for different user-requirements but still sharing the same Database.
Read more…

While developing applications on Google App Engine, you might need to find out total number of an Entity type being stored in the underlying Datastore. For people like me, coming from RDBMS background, it means one line query statement :

SELECT COUNT(*) from TABLE_NAME;

People starting with GAE might be surprised to know that we don’t have anything similar in GQL. The reason behind this limitation is the way Google Big Table (underlying storage for Datastore) is structured. It doesn’t support row counts as a fundamental concept. The only way for Datastore to return the result for us would be to run through the entire application data-set and execute query for finding matching entities. Following Java code will give us the desired result:

import com.google.appengine.api.datastore.Query;
		Query query = new Query("Entity Name");
		DatastoreService datastoreService = DatastoreServiceFactory.getDatastoreService();
		List<Entity> allEntities = datastoreService.prepare(query).asList(FetchOptions.Builder.withDefaults());
                return allEntities.size();

Some of the potential issues with the above code are:

• As the Datastore will execute queries to find matching entities, it will result in degrading performance as the number of entities increase over a period of time.
• All the Datastore operations are charged. If we are going to require this information quite frequently, it might adversely impact our final bills to be paid to Google and even more with new Pricing model to be effective soon.
• There is a hard limit of 30 seconds on very request to be executed. In case, we are dealing with large number of entities, it may just not work at all.

Following are a couple of different approaches for the problem:
Read more…

I have been discussing about capabilities of Google Fusion Tables in past couple of blogs. Continuing with the topic, I would like to talk about Heat Maps with Fusion Tables.

What is Heat Map For people who are not aware, the definition from Wikipedia says “A Heat Map is a graphic representation of the values taken by a variable from a table and represented as Colors”. A Heat Map uses different colors to represent the relative intensity of the variable. As an example, areas with low variable occurrence can be represented with Green, moderate with Yellow and high occurrence with Red color. Heat Maps along with location aware data can come as very handy way for representing data changes across different regions and time-lines. Following are some of the very popular use-cases of Heat Maps:

• Population Layout
• Voter behavior during Elections
• Incidence Occurrences
• Activity Levels

Of-course the usage is only limited by your imagination.

How to generate Heat Map with Google Fusion Table Data As mentioned in my previous posts as well, Fusion Tables allows you to add a layer over top of Google Map. For that, your data needs to have location information as well. Let’s take an example from publicly available data about Earth Quakes (4.5+ magnitude) happened anywhere in the World since 1973. A snapshot of the data is :

year,month,day,time,location,mag,depth
1973,01,01,034609.80,"-9.214, 150.634",5.3,41
1973,01,01,052229.80,"-15.012,-173.958",5.0,33
1973,01,01,092857.20,"-22.161, -65.792",4.8,205
1973,01,01,114237.50,"-35.513, -16.211",6.0,33

You can access the complete table data from here. Generating Heat Map from this table data using Fusion Tables is very simple:
Read more…

Few days back, I spent hours and hours trying to figure out why my web application was not getting deployed on Google App Engine. To save others from similar trouble, I would like to mention the reason behind the problem in my case.

Problem Context Mine was a Java based web application but the language and the contents are really not important in this case. When I was running the application in local GAE environment, everything was working perfectly fine. There were couple of versions already deployed on Google App Engine and they were also accessible. I was owner of the application, so there were no issues from permissions side as well. But while deploying the latest version update on Google App Engine, I was consistently getting 500 Internal Server Error. The detailed error message was :

Unable to update app: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=itj-helloworld&version=4.0.0&
500 Internal Server Error

Server Error (500)
A server error has occurred.

In my previous post, I gave a broad overview of Google Fusion Tables along with a brief Getting Started Guide. Continuing on the same subject, I would like to delve a bit deeper and discuss about Fusion Tables API.

Need of API As discussed in my earlier post, Fusion Tables is a Data Storage on Google Cloud. It allows us to store, share and execute SQL like queries on our data residing on Fusion Tables. We can of-course represent this data in multiple formats on Google Maps. As we saw in my previous post, publishing the contents of Fusion Tables data on a Google Map is quite easy. But more often than not, we would be dealing with dynamically changing data. For the Map to render the latest information, we will not be able to go through the route of importing tables at regular intervals. We would rather do these updates behind the scenes and let everyone have an up-to-date information all the time. That’s where Fusion Table API comes into picture.
Read more…

In my last post, I shared our experiences about building a large organization’s Intranet on Google App Engine using Vosao as underlying CMS. During that time, I touched upon Vosao’s extensible architecture which lets developers easily extend it’s functionality through plugin-based framework. In this post, I would like to talk about one of the plugins, we developed for our Intranet application.

Our requirement was that only authenticated users should be able to access the Intranet. Once that’s done, we should be displaying a Welcome message with connected user’s name and also provide a log-out URL at the top of all the pages. Fairly standard requirements, we would say for any company’s intranet but Vosao is designed to develop any interactive web-sites. It doesn’t impose any authentication mechanism for accessing the web-application and this seems logical as well. The decision about whether and which type of authentication is required is completely dependent on the application we are trying to develop.
Read more…

Lot of us use Google or other Map implementation for either finding a location, getting directions from one place to another or similar other purposes quite regularly. On a different note but very similar in terms of frequency, we often work with data, in-fact lots of data in our day-to-day work and would like to share and present it to people. The data is generally available to us in form of XLS spreadsheets, CSV files and back-end data-bases. Presentation of this data to a varied group of people may become tedious who may want to look and analyze it in different ways. Would it not be nice, if there is a way to store and present our data online in different formats. Even better (provided the data has geographical association), if we can over-lay it on a Map.

Google has come up with a solution to these issues in form of Google Fusion Tables. Here are some of its offerings:

• Upload location aware data on Fusion Tables and publish on Google Maps
• Represent in different Chart forms (Bar, Line etc)
• Represent in form of Motion and TimeLines
• Represent in form of HeatMap and Intensity Maps

Hello World on Google Fusion Tables Let’s start with a simple scenario where a Company is having multiple offices around the World, with information about the employee strength and revenue details from each of these regional offices. We would like this information to be displayed on Google Map.
Read more…

We have been working on Google App Engine right from its early stages and have developed several Web applications of various complexities on the platform. Recently we got a request from one of very large enterprise clients to port their existing (in-premise hosted) Intranet over to Google App Engine. As they had already decided to move their existing mail infrastructure to Google Apps, it was logical for them to leverage the benefits of Google App Engine for their Intranet as well.

In most of the scenarios, Company Intranet servers as a placeholder for sharing information among the employees. The information can be about new projects, upcoming events, quarterly/annual reports, birthdays, training calendar etc. and it generally keeps on changing at regular intervals. It would be ideal to have a CMS(Content Management System) embedded inside our web application. On one hand, Google App Engine would provide us the standard benefits of PaaS cloud while CMS would let us quickly and easily edit the contents without going through build and deployment cycles. For CMS, our preference was for Open Source solution for reasons of cost and also the possibility to modify the source to suit our requirements. During our search we came across following options:

Java based

• Vosao
• claymus
• Jerba CMS
• Richer CMS

Python based

• gutecms
• django-cms
• app-engine-site-creator

Our client’s technical team is well-versed with Java and for long-term maintenance reasons, we decided to stick to Java based solutions. After evaluating all Java based options, we decided to use Vosao for building our Intranet. Below are some of the reasons for the decision:
Read more…

I was one of the speaker of the second IndicThreads conference held at Pune on 3-4th June 2011.

Sessions at the conference dealt with key topics like Cloud Security, Amazon Elastic Beanstalk, Legal Issues in Cloud Computing, OpenStack, Xen Cloud Platform, Rails and CouchDB on the cloud, CloudFoundry, Gigapaces PAAS, Monitoring Cloud Applications, ORM with Objectify-Appengine, Scalable Architecture on Amazon AWS Cloud, Cloud Lock-in, Cloud Interoperability, Apache Hadoop, Map Reduce and Predictive Analysis.

My talk focussed on managing persistence on GAE. It dealt with choices available to a developer and then focussed on doing it with Objectify-Appengine.

Demo application is present here for download. For the instructions on how to run it please read the wiki.

Recently, Inphina started exploring Cloud Foundry to add onto its cloud offerings. The initial reactions have varied from good to very good to but ‘could they have this…’ In general it looks like that Cloud Foundry (CF) would be disrupting the PaaS market eventually. Why? Primarily because of the following reasons

• It is not tied to a particular environment or framework like .Net for Azure, Java/Python for the App Engine, Java for Beanstalk. CloudFoundry does support Java, Ruby, Grails, Node.js for now and the support for other languages / frameworks is on the roadmap.
• No vendor lock-in and no cloud lock-in. That is right you can deploy on any cloud. CF PaaS can be deployed on Amazon.
• Choice of Public and Private cloud infrastructures. This is one of the uphill battle that we are fighting with GAE at many enterprises. CloudFoundry PaaS can be deployed in your private datacenter and VMWare has enterprise support for that as well.
• Open PaaS, so take the code and build on top of that and contribute back.
• Backed by VMware and Spring (bought by VMware a couple of years back) which are known names in the enterprises. (I remember the day in earlier life when I wanted to bundle the Spring jar with my project and had to seek scores of approvals from the NY based financial house, but now no-one would blink an eyelid for Spring)

So does it mean that GAE and Azure would have to start losing sleep. Well, not at the moment because CF is still in beta. But, given the fact that GAE has partial support for Java and that Azure would be supporting .Net primarily (now that their Java offering has kind of fallen flat) the Open PaaS strategy of CF is intimidating. It enables developers to do what they do best i.e. develop in the language and framework of their choice and have little concerns about deployment environments.

One area where I feel still a lot of catch up is required is services. GAE as a PaaS offers a bouquet of services which are unmatched by any other PaaS. At least, at the moment. For a rapid development and prototyping approach I would still probably fall back to GAE where I get mails, namespaces, cache, channel api, oauth etc etc. Though CF could easily provide administration services with Hyperic and system level services with GemFire and RabbitMQ but other application level services which GAE can provide are still missing.

All in all, an Open PaaS which needs to followed closely. Enjoy the presentation …

Inphina, as an expert on Google App Engine and Google Apps has helped more than 10 medium to large organizations help take advantage of the cloud by building, migrating or re-engineering their complex line of business applications to the cloud thereby making significant reductions in their capex expenditure. If you are interested in an assessment send us a mail at cloud@inphina.com