Festival and Holiday season is all around us. Many of us are already on holidays, taking well deserved break and having fun with family and friends. This is also a time to sit back, take a look how things went in past 12 months and plan for the coming year.
For Inphina, it has been an eventful year. We did projects on lot of new and diverse technologies like Google Fusion Tables, Maps, Google App Engine, Lift, Scala and lot more. Our partners list also increased substantially this year and we got opportunity to work with organizations from Singapore, Australia, Germany, UK, USA and of-course India. We started with CSD (Certified Scrum Developer) program and were the first one to execute it in India. Along with many successful public and private courses in India, we were able to take it outside India and did our first International CSD course in Kathmandu, Nepal. Most importantly, we had great fun and learning all through-out the year. We would like to thank all our partners for their confidence and support.
On behalf of everyone at Inphina, we would like to wish you and your dear ones a Merry Christmas and happy times in the year ahead.
We have been working on Google App Engine since its early days and have developed and ported several applications on the platform for our clients. One of the major hinderance for our Enterprise clients for not being able to move to GAE has been Datastore being the only storage option available till date. Major reasons for these enterprises not using Datastore are:
- (In case of migration) Substantial efforts in adapting the application code base to Google Datastore specifications
- Potentially huge data-migration efforts
- Vendor Lock-In as both the application code and data residing in Datastore can’t be taken to an in-premise Infrastructure or another cloud platform seamlessly
Keeping all these requests and potential business opportunities in mind, Google has recently announced Cloud SQL.
What is Google Cloud SQL
- It’s a Relational Database on Cloud offering from Google.
- Offers MySQL Database environment with JDBC support for Java applications and DB-API support for Python applications.
- Being a service on Cloud, it offers traditional advantages of any Cloud service like no maintenance, administration, replication and backup efforts
- To ensure high-availability, data gets replicated across multiple geographic regions
- Easy to use Web-Interface for managing multiple Instances
Google Cloud SQL & Google App Engine
A very good integration has been provided for your applications running on Google App Engine to use Google Cloud SQL as back-end storage mechanism. Furthermore, one SQL instance can be shared across multiple Google App Engine applications. This feature should be quite helpful as often we have a family of Web Applications for different user-requirements but still sharing the same Database.
I have been using Subversion as source code version control tool since many years. Though I have used Mercurial for couple of hobby projects earlier but few weeks back, I started using it on my first Industrial project. Here, we are working on multiple feature-sets which need to go into Production at different time-lines. The best way to manage such scenarios is the use of branches. The concept and way of working with branches in Mercurial is quite different from Subversion. Here are few commands which one might find helpful while starting to work with Mercurial branches:
hg branches //shows all the branches present in the repository
There is always at-least one branch in Mercurial Repository which is called default as compared to trunk in Subversion.
hg branch new_branch_name //creates a new named branch
Named branches help in easy identification of the work being done on that branch.
hg commit //commits all the changes locally to the newly created branch hg push --new-branch //pushes all the changes of this branch along with the branch to mercurial server
Only when we have committed and pushed our changes in newly created branch, other team members will be able to see the branch and our changes. Once a team member has committed the changes in a branch and other people would like to do further work on the same branch, they can use the following flow:
hg pull //gets all the changes on default as well as other branches hg update -C branch_name //switches to the branch
On regular intervals, we will need to merge our changes from one branch to other for integrating different feature-sets. Following are the commands for the purpose
hg merge default //merges all the changes from default branch to your current working branch hg commit //commits all the changes in your current working branch hg push //pushes all the change-sets into the repository for other members to use them
For more detailed information on branching, please refer to branching tutorial available on Mercurial site.
While developing applications on Google App Engine, you might need to find out total number of an Entity type being stored in the underlying Datastore. For people like me, coming from RDBMS background, it means one line query statement :
SELECT COUNT(*) from TABLE_NAME;
People starting with GAE might be surprised to know that we don’t have anything similar in GQL. The reason behind this limitation is the way Google Big Table (underlying storage for Datastore) is structured. It doesn’t support row counts as a fundamental concept. The only way for Datastore to return the result for us would be to run through the entire application data-set and execute query for finding matching entities. Following Java code will give us the desired result:
import com.google.appengine.api.datastore.Query; Query query = new Query("Entity Name"); DatastoreService datastoreService = DatastoreServiceFactory.getDatastoreService(); List<Entity> allEntities = datastoreService.prepare(query).asList(FetchOptions.Builder.withDefaults()); return allEntities.size();
Some of the potential issues with the above code are:
- As the Datastore will execute queries to find matching entities, it will result in degrading performance as the number of entities increase over a period of time.
- All the Datastore operations are charged. If we are going to require this information quite frequently, it might adversely impact our final bills to be paid to Google and even more with new Pricing model to be effective soon.
- There is a hard limit of 30 seconds on very request to be executed. In case, we are dealing with large number of entities, it may just not work at all.
Following are a couple of different approaches for the problem:
What is Heat Map For people who are not aware, the definition from Wikipedia says “A Heat Map is a graphic representation of the values taken by a variable from a table and represented as Colors”. A Heat Map uses different colors to represent the relative intensity of the variable. As an example, areas with low variable occurrence can be represented with Green, moderate with Yellow and high occurrence with Red color. Heat Maps along with location aware data can come as very handy way for representing data changes across different regions and time-lines. Following are some of the very popular use-cases of Heat Maps:
- Population Layout
- Voter behavior during Elections
- Incidence Occurrences
- Activity Levels
Of-course the usage is only limited by your imagination.
How to generate Heat Map with Google Fusion Table Data As mentioned in my previous posts as well, Fusion Tables allows you to add a layer over top of Google Map. For that, your data needs to have location information as well. Let’s take an example from publicly available data about Earth Quakes (4.5+ magnitude) happened anywhere in the World since 1973. A snapshot of the data is :
year,month,day,time,location,mag,depth 1973,01,01,034609.80,"-9.214, 150.634",5.3,41 1973,01,01,052229.80,"-15.012,-173.958",5.0,33 1973,01,01,092857.20,"-22.161, -65.792",4.8,205 1973,01,01,114237.50,"-35.513, -16.211",6.0,33
Few days back, I spent hours and hours trying to figure out why my web application was not getting deployed on Google App Engine. To save others from similar trouble, I would like to mention the reason behind the problem in my case.
Problem Context Mine was a Java based web application but the language and the contents are really not important in this case. When I was running the application in local GAE environment, everything was working perfectly fine. There were couple of versions already deployed on Google App Engine and they were also accessible. I was owner of the application, so there were no issues from permissions side as well. But while deploying the latest version update on Google App Engine, I was consistently getting 500 Internal Server Error. The detailed error message was :
Unable to update app: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=itj-helloworld&version=4.0.0& 500 Internal Server Error Server Error (500) A server error has occurred.
In my previous post, I gave a broad overview of Google Fusion Tables along with a brief Getting Started Guide. Continuing on the same subject, I would like to delve a bit deeper and discuss about Fusion Tables API.
Need of API As discussed in my earlier post, Fusion Tables is a Data Storage on Google Cloud. It allows us to store, share and execute SQL like queries on our data residing on Fusion Tables. We can of-course represent this data in multiple formats on Google Maps. As we saw in my previous post, publishing the contents of Fusion Tables data on a Google Map is quite easy. But more often than not, we would be dealing with dynamically changing data. For the Map to render the latest information, we will not be able to go through the route of importing tables at regular intervals. We would rather do these updates behind the scenes and let everyone have an up-to-date information all the time. That’s where Fusion Table API comes into picture.
In my last post, I shared our experiences about building a large organization’s Intranet on Google App Engine using Vosao as underlying CMS. During that time, I touched upon Vosao’s extensible architecture which lets developers easily extend it’s functionality through plugin-based framework. In this post, I would like to talk about one of the plugins, we developed for our Intranet application.
Our requirement was that only authenticated users should be able to access the Intranet. Once that’s done, we should be displaying a Welcome message with connected user’s name and also provide a log-out URL at the top of all the pages. Fairly standard requirements, we would say for any company’s intranet but Vosao is designed to develop any interactive web-sites. It doesn’t impose any authentication mechanism for accessing the web-application and this seems logical as well. The decision about whether and which type of authentication is required is completely dependent on the application we are trying to develop.
Lot of us use Google or other Map implementation for either finding a location, getting directions from one place to another or similar other purposes quite regularly. On a different note but very similar in terms of frequency, we often work with data, in-fact lots of data in our day-to-day work and would like to share and present it to people. The data is generally available to us in form of XLS spreadsheets, CSV files and back-end data-bases. Presentation of this data to a varied group of people may become tedious who may want to look and analyze it in different ways. Would it not be nice, if there is a way to store and present our data online in different formats. Even better (provided the data has geographical association), if we can over-lay it on a Map.
Google has come up with a solution to these issues in form of Google Fusion Tables. Here are some of its offerings:
- Upload location aware data on Fusion Tables and publish on Google Maps
- Represent in different Chart forms (Bar, Line etc)
- Represent in form of Motion and TimeLines
- Represent in form of HeatMap and Intensity Maps
Hello World on Google Fusion Tables Let’s start with a simple scenario where a Company is having multiple offices around the World, with information about the employee strength and revenue details from each of these regional offices. We would like this information to be displayed on Google Map.
We have been working on Google App Engine right from its early stages and have developed several Web applications of various complexities on the platform. Recently we got a request from one of very large enterprise clients to port their existing (in-premise hosted) Intranet over to Google App Engine. As they had already decided to move their existing mail infrastructure to Google Apps, it was logical for them to leverage the benefits of Google App Engine for their Intranet as well.
In most of the scenarios, Company Intranet servers as a placeholder for sharing information among the employees. The information can be about new projects, upcoming events, quarterly/annual reports, birthdays, training calendar etc. and it generally keeps on changing at regular intervals. It would be ideal to have a CMS(Content Management System) embedded inside our web application. On one hand, Google App Engine would provide us the standard benefits of PaaS cloud while CMS would let us quickly and easily edit the contents without going through build and deployment cycles. For CMS, our preference was for Open Source solution for reasons of cost and also the possibility to modify the source to suit our requirements. During our search we came across following options:
Our client’s technical team is well-versed with Java and for long-term maintenance reasons, we decided to stick to Java based solutions. After evaluating all Java based options, we decided to use Vosao for building our Intranet. Below are some of the reasons for the decision: