Attribute vs Identity
Red black pushes
AutoScaling
Tuesday, December 20, 2011
Tuesday, November 29, 2011
Relational DBs vs NoSQL vs OLAP(Hadoop) - When to use What
There are hundreds of database technologies and there are more and more evolving day by day - generally, they could be classified into these three categories -
Relational DBs
These are our old school relational dbs that have worked for us wonderfully for the last 3 or more decades.
Oracle,
MySQL,
SQL Server,
Informix
are under this category. These dbs are meant mainly for transactional processing (OLTP). Although, a few years ago, people largely used one of these according to their needs, with the evolution of NoSQL databases, there needs to be all the more reason to differentiate when to use relational DBs vs the NoSQL DBs. If your business need is to involve transactional processing/processing credit cards, customer accounts which are highly sensitive, relational DB is still the best way to go. The ACID properties (Atomicity, consistency, isolation, durability) draws main attention here since this guarantees that data is processed reliably.
NoSQL DBs
This is the new kid in town and so is drawing a lot of attention during the past few years. Some of the names that you might have heard in this category are -
- Cassandra
- MongoDB
- Apache Couch DB
- Simple DB (from Amazon's cloud offering)
- MarkLogic
- Riak
Most of these DBs mainly are implemented with the CAP (Consistency, Availability, Partition Tolerance) theorem in mind. And what is the need for a NoSQL db? This comes in handy when you need to process volumes of data efficiently without caring about the transactional aspect of the business. The data in these systems are for the most part stored in the form of key-value pairs.
One interesting thing that is noticeable is that all of these dbs call their table-equivalents using different names.
In Cassandra, the table-equivalent is called a column family. Names are always intriguing. May be it is a family of columns and that is why it is being called as a column family. In that case, why not a row family?:-)
OLAP (Hadoop)
These days, it is all about Big data/Hadoop. More on this soon.
Relational DBs
These are our old school relational dbs that have worked for us wonderfully for the last 3 or more decades.
Oracle,
MySQL,
SQL Server,
Informix
are under this category. These dbs are meant mainly for transactional processing (OLTP). Although, a few years ago, people largely used one of these according to their needs, with the evolution of NoSQL databases, there needs to be all the more reason to differentiate when to use relational DBs vs the NoSQL DBs. If your business need is to involve transactional processing/processing credit cards, customer accounts which are highly sensitive, relational DB is still the best way to go. The ACID properties (Atomicity, consistency, isolation, durability) draws main attention here since this guarantees that data is processed reliably.
NoSQL DBs
This is the new kid in town and so is drawing a lot of attention during the past few years. Some of the names that you might have heard in this category are -
- Cassandra
- MongoDB
- Apache Couch DB
- Simple DB (from Amazon's cloud offering)
- MarkLogic
- Riak
Most of these DBs mainly are implemented with the CAP (Consistency, Availability, Partition Tolerance) theorem in mind. And what is the need for a NoSQL db? This comes in handy when you need to process volumes of data efficiently without caring about the transactional aspect of the business. The data in these systems are for the most part stored in the form of key-value pairs.
One interesting thing that is noticeable is that all of these dbs call their table-equivalents using different names.
In Cassandra, the table-equivalent is called a column family. Names are always intriguing. May be it is a family of columns and that is why it is being called as a column family. In that case, why not a row family?:-)
OLAP (Hadoop)
These days, it is all about Big data/Hadoop. More on this soon.
Friday, November 18, 2011
Building an API
Why do need them
Evaluating Technologies
Design
Feedback loop
Versioning
Supporting the customers
Best Practices
Scaling your system
Consumers
Evaluating Technologies
Design
Feedback loop
Versioning
Supporting the customers
Best Practices
Scaling your system
Consumers
Storm: Realtime Dataprocessing at twitter
Here is Nathan Marz's blog with all the details - http://nathanmarz.com/
Tuesday, August 9, 2011
Today's topic is Avro
Apache Avro™ is a data serialization system. To know more about Avro, visit the URL : http://avro.apache.org/docs/current/
Tuesday, August 2, 2011
Cloud computing
Cloud Computing is where virtual/utility computers/resources are sprung up on demand to perform services required to run an application.
Use case:
you might have a real web application that needs to be hosted. But the load/traffic on this site is not constant. Certain times based on an Event, the traffic surges - and other times its a trickle. Cloud Computing using say Amazon EC2 comes to your rescue here. You essentially create an “Amazon Machine Instance” (AMI) - with the code/software required to run your application. You store this “image” on a Storage service - such as Amazon S3. Then, you just launch these images/apps as and when you require them. You can have 0 or 1 instance of your application running at any given time and when the traffic flow increases, you deploy a few more on the go. You pay by the amount of time these machines were running and the bandwidth consumed - well roughly speaking.
Players
Amazon EC2 currently seems to be the market leader in terms of both the buzz and maturity. Other players include Google AppEngine, Sun microsystems, Microsoft etc. If you are looking for a brief comparision of their offerings, this page here could help -> http://weblog.infoworld.com/tcdaily/archives/2008/07/video_tours_of.html?source=fssr.
Potential Customers
Startups
Using a on-demand cloud hosting is good idea for startups.
For one, you dont have to invest in hardware until you know what your scalability requirements and yes, its different from leasing a cage at a datacenter where you have total control, but its so much more better on your wallet.
Small Business
Well, at this point you are more “stable” and know what your load and hosting requirements are. It still might be valuable to host your application in the “cloud”. But your mileage may vary vis-a-vis leasing a cage. Note that this is mostly in terms of the cloud (such as EC2) as a hosting platform.
Mid-Large Companies
For large enterprises who already have an established IT organization, “Cloud Computing” still provides value in 2 circumstances :
1) As an “Overflow Buffer”: Enterprise no longer need to over-provision their data center for the peak load.
They just need to install equipment in-house for the average load. When peak load arrives, they just automatically provision additional resource from the cloud and redirect the excess load to those extra resource. This means huge equipment cost savings.
2) As an “Experimental Playground”: Enterprise need a fast turn around to test out new ideas and customer acceptance. But it is hard to justify equipment purchase before the idea is proven. So they can deploy the new project in the cloud to test out the acceptance. If it doesn’t work, they just tear the project down and no equipment is wasted. If the idea work, they start to purchase in-house equipment and migrate the application from the cloud to the inhouse data center.
For these large enterprise, their ultimate operating environment can be a mix of “data-center” and “public cloud”. One major challenge is how to migrate their application components seamlessly between the “private” and “public” facilities efficiently without sacrificing security, reliability and performance. This is an area that Amazon/Google/Microsoft are not motivated to solve (as Amazon/Google don’t like your datacenter portion and Microsoft won’t take care of your enterprise Java apps). This is an area in which Startups can step in.
Financial/Private/Secure Companies
For these institutions, the data security and handling is of paramount importance. Amazon Ec2 for instance does not provide SLAs except on a very few AWS APIs. For e.g. whats the garauntee on data stored in the S3? What is the round trip garauntee for data access from a SimpleDB? There are no response time garantees. This is going to be a challenge. It might make more sense for this sector to invest in “private clouds”.
Gartner has predicated “Private cloud networks are the future of corporate IT” at http://www.networkworld.com/news/2008/111208-private-cloud-networks.html
Now that we went through the definition of the cloud, the players and who it can benefit, its clear that Cloud Computing is here to stay. Which means to say that there is a going to be a definite impact on the Software Industry.
For e.g.
Licensing model - how does one deal with a “per cpu” license? In the cloud, resources are booted up on demand - sometimes in 100s. What about Open Source licences?
Startup opportunities.
There will be opportunity for app/service vendors to build standardization, virtualization and services on top of existing cloud infrastructure.
Create Hosting Software Stacks (e.g. AMIs)
Use case:
you might have a real web application that needs to be hosted. But the load/traffic on this site is not constant. Certain times based on an Event, the traffic surges - and other times its a trickle. Cloud Computing using say Amazon EC2 comes to your rescue here. You essentially create an “Amazon Machine Instance” (AMI) - with the code/software required to run your application. You store this “image” on a Storage service - such as Amazon S3. Then, you just launch these images/apps as and when you require them. You can have 0 or 1 instance of your application running at any given time and when the traffic flow increases, you deploy a few more on the go. You pay by the amount of time these machines were running and the bandwidth consumed - well roughly speaking.
Players
Amazon EC2 currently seems to be the market leader in terms of both the buzz and maturity. Other players include Google AppEngine, Sun microsystems, Microsoft etc. If you are looking for a brief comparision of their offerings, this page here could help -> http://weblog.infoworld.com/tcdaily/archives/2008/07/video_tours_of.html?source=fssr.
Potential Customers
Startups
Using a on-demand cloud hosting is good idea for startups.
For one, you dont have to invest in hardware until you know what your scalability requirements and yes, its different from leasing a cage at a datacenter where you have total control, but its so much more better on your wallet.
Small Business
Well, at this point you are more “stable” and know what your load and hosting requirements are. It still might be valuable to host your application in the “cloud”. But your mileage may vary vis-a-vis leasing a cage. Note that this is mostly in terms of the cloud (such as EC2) as a hosting platform.
Mid-Large Companies
For large enterprises who already have an established IT organization, “Cloud Computing” still provides value in 2 circumstances :
1) As an “Overflow Buffer”: Enterprise no longer need to over-provision their data center for the peak load.
They just need to install equipment in-house for the average load. When peak load arrives, they just automatically provision additional resource from the cloud and redirect the excess load to those extra resource. This means huge equipment cost savings.
2) As an “Experimental Playground”: Enterprise need a fast turn around to test out new ideas and customer acceptance. But it is hard to justify equipment purchase before the idea is proven. So they can deploy the new project in the cloud to test out the acceptance. If it doesn’t work, they just tear the project down and no equipment is wasted. If the idea work, they start to purchase in-house equipment and migrate the application from the cloud to the inhouse data center.
For these large enterprise, their ultimate operating environment can be a mix of “data-center” and “public cloud”. One major challenge is how to migrate their application components seamlessly between the “private” and “public” facilities efficiently without sacrificing security, reliability and performance. This is an area that Amazon/Google/Microsoft are not motivated to solve (as Amazon/Google don’t like your datacenter portion and Microsoft won’t take care of your enterprise Java apps). This is an area in which Startups can step in.
Financial/Private/Secure Companies
For these institutions, the data security and handling is of paramount importance. Amazon Ec2 for instance does not provide SLAs except on a very few AWS APIs. For e.g. whats the garauntee on data stored in the S3? What is the round trip garauntee for data access from a SimpleDB? There are no response time garantees. This is going to be a challenge. It might make more sense for this sector to invest in “private clouds”.
Gartner has predicated “Private cloud networks are the future of corporate IT” at http://www.networkworld.com/news/2008/111208-private-cloud-networks.html
Now that we went through the definition of the cloud, the players and who it can benefit, its clear that Cloud Computing is here to stay. Which means to say that there is a going to be a definite impact on the Software Industry.
For e.g.
Licensing model - how does one deal with a “per cpu” license? In the cloud, resources are booted up on demand - sometimes in 100s. What about Open Source licences?
Startup opportunities.
There will be opportunity for app/service vendors to build standardization, virtualization and services on top of existing cloud infrastructure.
Create Hosting Software Stacks (e.g. AMIs)
Friday, July 29, 2011
Tech blogs
Other Technology Blogs
* All Things D - http://allthingsd.com/
* AppleInsider - http://appleinsider.com/
* Ars Technica - http://arstechnica.com/
* Boy Genius Report - http://www.bgr.com/
* CNet News - http://news.cnet.com/
* DigiTimes - http://www.digitimes.com/index.asp
* Digits - http://blogs.wsj.com/digits/
* GigaOm - http://gigaom.com/
* Mashable - http://mashable.com/
* PaidContent - http://paidcontent.org/
* Post Tech - http://paidcontent.org/
* ReadWriteWeb - http://www.washingtonpost.com/blogs/post-tech
* Search Engine Land - http://searchengineland.com/
* Silicon Alley Insider - http://www.businessinsider.com/sai
* Tech Trader Daily - http://blogs.barrons.com/techtraderdaily/
* TechCrunch - http://techcrunch.com/
* Techdirt - http://www.techdirt.com/
* Techflash - http://www.techflash.com/
* Techmeme - http://techmeme.com/
* The Register - http://www.theregister.co.uk/
* VentureBeat - http://venturebeat.com/
* Wired News - http://www.wired.com/
* All Things D - http://allthingsd.com/
* AppleInsider - http://appleinsider.com/
* Ars Technica - http://arstechnica.com/
* Boy Genius Report - http://www.bgr.com/
* CNet News - http://news.cnet.com/
* DigiTimes - http://www.digitimes.com/index.asp
* Digits - http://blogs.wsj.com/digits/
* GigaOm - http://gigaom.com/
* Mashable - http://mashable.com/
* PaidContent - http://paidcontent.org/
* Post Tech - http://paidcontent.org/
* ReadWriteWeb - http://www.washingtonpost.com/blogs/post-tech
* Search Engine Land - http://searchengineland.com/
* Silicon Alley Insider - http://www.businessinsider.com/sai
* Tech Trader Daily - http://blogs.barrons.com/techtraderdaily/
* TechCrunch - http://techcrunch.com/
* Techdirt - http://www.techdirt.com/
* Techflash - http://www.techflash.com/
* Techmeme - http://techmeme.com/
* The Register - http://www.theregister.co.uk/
* VentureBeat - http://venturebeat.com/
* Wired News - http://www.wired.com/
Monday, July 25, 2011
Qualities of a Leader
- Intelligence
- Creativity
- Wisdom
- Charisma
- Drive to Succeed
- Hunger for innovation
- Willingness to challenge established ideas and practices
- Compulsive risk taker
- Novelty seeking behavior
- Discipline
- Fortitude
- Functions relating to pleasure and reward
- As a key motivator, pleasure is central to learning
Here are some details :
So, when searching for your organization’s next leader, look for someone with an attenuated dopamine function: someone who is never satisfied with the status quo, someone who wants the feeling of success more than others — but likes it less.
WHEN we think of the qualities we seek in visionary leaders, we think of intelligence, creativity, wisdom and charisma, but also the drive to succeed, a hunger for innovation, a willingness to challenge established ideas and practices.
But in fact, the psychological profile of a compelling leader — think of tech pioneers like Jeff Bezos, Larry Ellison and Steven P. Jobs — is also that of the compulsive risk-taker, someone with a high degree of novelty-seeking behavior. In short, what we seek in leaders is often the same kind of personality type that is found in addicts, whether they are dependent on gambling, alcohol, sex or drugs.
How can this be? We typically see addicts as weak-willed losers, and chief executives and entrepreneurs are people with discipline and fortitude. To understand this apparent contradiction we need to look under the hood of the brain, and in particular at the functions that relate to pleasure and reward.
As a key motivator, pleasure is central to learning
- Creativity
- Wisdom
- Charisma
- Drive to Succeed
- Hunger for innovation
- Willingness to challenge established ideas and practices
- Compulsive risk taker
- Novelty seeking behavior
- Discipline
- Fortitude
- Functions relating to pleasure and reward
- As a key motivator, pleasure is central to learning
Here are some details :
So, when searching for your organization’s next leader, look for someone with an attenuated dopamine function: someone who is never satisfied with the status quo, someone who wants the feeling of success more than others — but likes it less.
WHEN we think of the qualities we seek in visionary leaders, we think of intelligence, creativity, wisdom and charisma, but also the drive to succeed, a hunger for innovation, a willingness to challenge established ideas and practices.
But in fact, the psychological profile of a compelling leader — think of tech pioneers like Jeff Bezos, Larry Ellison and Steven P. Jobs — is also that of the compulsive risk-taker, someone with a high degree of novelty-seeking behavior. In short, what we seek in leaders is often the same kind of personality type that is found in addicts, whether they are dependent on gambling, alcohol, sex or drugs.
How can this be? We typically see addicts as weak-willed losers, and chief executives and entrepreneurs are people with discipline and fortitude. To understand this apparent contradiction we need to look under the hood of the brain, and in particular at the functions that relate to pleasure and reward.
As a key motivator, pleasure is central to learning
Subscribe to:
Posts (Atom)
