tag:blogger.com,1999:blog-24150011474939914072024-03-13T22:26:46.764+05:30Tech GoodiesDeepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.comBlogger57125tag:blogger.com,1999:blog-2415001147493991407.post-76951838679685504672021-04-11T00:12:00.001+05:302021-04-11T00:12:22.137+05:30Heroku Custom Trust Store for SSL Handshake<p> </p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-wQfShNSDsVI/YHHv24IWrQI/AAAAAAAAIWc/vpliZXiUj1sPG-BqGfPvc6fIYXH0SM4YwCLcBGAsYHQ/s880/heroku.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="372" data-original-width="880" src="https://1.bp.blogspot.com/-wQfShNSDsVI/YHHv24IWrQI/AAAAAAAAIWc/vpliZXiUj1sPG-BqGfPvc6fIYXH0SM4YwCLcBGAsYHQ/s320/heroku.png" width="320" /></a>
</div>
<br />
<p><br /></p>
<p>
Working with <a href="https://www.heroku.com/" target="_blank">Heroku</a> for
deploying apps (java, nodejs, etc..) is made very easy but while integrating
one of the service hosted on AWS, I was getting SSLHandshakeException and
could not figure it out initially how to resolve the issue.
</p>
<p>
Exception while making a request to the target system from the app hosted on
Heroku:
</p>
<p>
<span style="background-color: #f7f8fb; color: #3f3f44; font-family: consolas, monaco, monospace; font-size: 13px;"><br /></span>
</p>
<div style="text-align: left;">
<div>
<div>
<pre> <div>
<span style="background-color: #f3f3f3;">javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: </span></div><div><span style="background-color: #f3f3f3;">PKIX path building failed<br /></span></div><div><span style="background-color: #f3f3f3;">sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid </span></div><div><span style="background-color: #f3f3f3;">certification path to requested target</span></div><div> </div>
</pre>
</div>
</div>
</div>
<p>
Heroku allows us to
<a href="https://devcenter.heroku.com/articles/customizing-the-jdk" target="_blank">customise the JDK</a>. To Add a custom trust store
</p>
<p>
1) Download the base keystore (cacert) from following link<span style="background-color: #f7f8fb; color: #3f3f44; font-family: consolas, monaco, monospace; font-size: 13px;"> </span>
</p>
<p>
<a href="https://heroku-cacerts.herokuapp.com/heroku_cacerts" style="background: rgb(255, 255, 255); box-sizing: border-box; color: #006bd4; font-family: BentonSans, "Helvetica Neue", helvetica, arial, sans-serif; font-size: 16px; text-decoration-line: none;">download the base Heroku keystore</a>
</p>
<p>2) Rename downloaded file to cacerts (heroku_cacerts to cacerts)</p>
<p>
3) Add the certificate (.pem) file of the target system by executing the
following
</p>
<div style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px; text-align: left;"><br /></div>
<div style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px; text-align: left;"> <pre><span style="background-color: #f3f3f3;"> keytool -import -keystore /Users/dsinghvi/Downloads/cacerts -file
/Users/dsinghvi/Downloads/target_system_certifacte.pem</span>
</pre>
</div>
<p class="p1" style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;"><br /></p>
<p class="p1" style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<span style="font-family: Times; font-size: medium;"><br /></span>
</p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<span style="font-family: Times; font-size: medium;">4) Add the cacerts file file to you project as follows (in the
folder </span><PROJET_DIR_PATH>/.jdk-overlay/jre/lib/security/<span style="font-family: Times; font-size: medium;">)</span>
</p>
<p class="p1" style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<span style="font-family: Times; font-size: medium;"><br /></span>
</p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<span style="font-family: Times; font-size: medium;"></span>
</p>
<div class="separator" style="clear: both; text-align: center;"><br /></div>
<br /><br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="font-family: "Helvetica Neue"; font-size: 12px; margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;">
<a href="https://1.bp.blogspot.com/-LklvMWY8NXs/YHHolVl2LPI/AAAAAAAAIWM/wM-JEKa1l30LS7pSx21wc11wkTEmjn3lgCLcBGAsYHQ/s287/customjdk_cacerts.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="243" data-original-width="287" src="https://1.bp.blogspot.com/-LklvMWY8NXs/YHHolVl2LPI/AAAAAAAAIWM/wM-JEKa1l30LS7pSx21wc11wkTEmjn3lgCLcBGAsYHQ/s0/customjdk_cacerts.png" /></a>
</td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">
<span style="font-size: 12px;"><br />
<div style="text-align: left;"><br /></div>
<div style="text-align: left;"><br /><br /></div></span>
</td>
</tr>
</tbody>
</table>
<p style="clear: both; text-align: center;"></p>
<p style="font-family: "Helvetica Neue"; font-size: 12px; text-align: left;">
<br />
</p>
<p></p>
<div class="separator" style="clear: both; font-family: "Helvetica Neue"; font-size: 12px; text-align: center;">
<br />
</div>
<p></p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
5) Git add, commit and push to Heroku app
</p>
<div style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px; text-align: left;"><br /></div><pre><div style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px; text-align: left;"><span style="background-color: #f3f3f3;">git add .jdk-overlay/jre/lib/security/cacerts
</span></div><div style="text-align: left;"><span style="background-color: #f3f3f3;">git commit -m "custom trust store for certificate issue"
</span></div><div style="text-align: left;"><span style="background-color: #f3f3f3;">git push heroku master</span></div></pre><div><br /></div>
<div>
<p style="font-family: "Helvetica Neue"; font-size: 12px;">
<span style="font-family: Times; font-size: medium;">5) Optionally verify if cacerts has been updated</span>
</p>
<p style="font-family: "Helvetica Neue"; font-size: 12px;">
<span style="font-family: Times; font-size: medium;">Added cacerts would be overridden in the /app/.jdk/jre/lib/security
(jdk1.8) and /app/.jdk/lib/security (JDK 1.9 and above)</span>
</p>
<p style="font-family: "Helvetica Neue"; font-size: 12px;">
<span style="font-family: Times; font-size: medium;">This can be verified in the app deployed using:</span>
</p>
<p style="font-family: "Helvetica Neue"; font-size: 12px;"><br /></p>
<div style="font-family: "Helvetica Neue"; font-size: 12px; text-align: left;"><pre><span style="background-color: #f3f3f3; font-family: Times; font-size: medium;">heroku run bash --app <APP_NAME></span>
</pre>
</div>
<p style="font-family: "Helvetica Neue"; font-size: 12px;"><br /></p>
</div>
<div>
<p class="p1" style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<span style="font-family: Times; font-size: medium;">5) Optionally if you want to verify whether trust store has been updated
or not </span>
</p>
<p class="p1" style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<span style="font-family: Times; font-size: medium;"><br /></span>
</p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
Before importing target system certificate</p><div style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px; text-align: left;"><pre><span style="background-color: #f3f3f3;">keytool -list -v -keystore /Users/dsinghvi/Downloads/cacerts > cert1.txt</span>
</pre></div>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
After importing target system certificate
</p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;"> </p><pre><span style="background-color: #f3f3f3;">keytool -list -v -keystore /Users/dsinghvi/Downloads/cacerts > cert2.txt</span>
</pre>
<p></p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<br />
</p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
You would find the difference between cert1.txt and cert2.txt as follows:
</p>
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<br />
</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-A6uAArdelcs/YHHp1-Nuv0I/AAAAAAAAIWU/FKtDYTLBpQcln6tPa7DG3nu01IXsUnT8wCLcBGAsYHQ/s1376/cacertsupdated.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="235" data-original-width="1376" height="112" src="https://1.bp.blogspot.com/-A6uAArdelcs/YHHp1-Nuv0I/AAAAAAAAIWU/FKtDYTLBpQcln6tPa7DG3nu01IXsUnT8wCLcBGAsYHQ/w649-h112/cacertsupdated.png" width="649" /></a>
</div>
<br />
<p class="p1" style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">
<br />
</p>
</div>
<p class="p1" style="font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;"></p>
<div class="separator" style="clear: both; text-align: center;"><br /></div>
<div class="separator" style="clear: both; text-align: center;"><br /></div>
<div class="separator" style="clear: both; text-align: center;"><br /></div>
<div class="separator" style="clear: both; text-align: center;"><b>
Happy Coding
</b></div>
<br />
<p></p>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0Bengaluru, Karnataka, India12.9715987 77.5945627-15.338635136178846 42.4383127 41.281832536178847 112.7508127tag:blogger.com,1999:blog-2415001147493991407.post-54347530254579898392021-04-10T23:21:00.002+05:302021-04-10T23:21:25.076+05:30Impact of Covid for Supply Chain<p> I was sharing my thoughts on the impact of Covid-19 and the new opportunity in supply chain domain.</p>
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="411" src="https://www.youtube.com/embed/kWXEPxsDFIA" width="494" youtube-src-id="kWXEPxsDFIA"></iframe></div><p><br /></p>
<div class="separator" style="clear: both; text-align: center;"><br /></div><p><br /></p>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-5242838140857638832020-04-08T20:41:00.000+05:302020-06-12T11:47:25.114+05:30DB Multi-Tenancy Modelling for Microservice <div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Multi-Tenancy with the SaaS based model does not bring any change on the application layer as multiple services deployment of application would be able to accommodate the
load on application but it would matter how do we decide the modelling of the database w.r.t. Multi-Tenancy.
</span><br />
<br />
Following are the factors to be considered for decision making for multi-tenancy:<br />
<br />
<b>Cost </b>Cost per tenants if separate DB/SCHEMA is selected per tenants.<br />
<br />
<b>Performance</b>
Application performance & scalability would impact based on the number of tenants, data size per tenants and the workload.<br />
<br />
<b>Time</b>
Development & Release migration efforts when multiple databases/schemas are in production. Complexity for managing the operations for maintaining multiple DBs.
<br />
<br />
<span style="font-size: large;"><br /></span>
<span style="font-size: large;"><br /></span>
<span style="font-size: large;">Tenancy Modelling
</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><i>Database per Tenant</i></b>
Single Tenant Architecture where each tenant has its own database and is isolated from other tenants.
</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><i>Shared Database</i></b>
Multi Tenant Architecture with separate schema where all tenants share a database, but have their own database schemas and tables.
</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><i>Shared Database and Shared Schema</i></b>
Multi Tenant Architecture with all tenants share a database and tables. Every table has a Column with the Tenant Identifier, that shows the tenant of the row.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://1.bp.blogspot.com/-6ZOZxRu7e60/XotBTpWl0YI/AAAAAAAAH_s/jB7tlLy_Jiw9LTy24nd4B80a1DleydT_wCLcBGAsYHQ/s1600/tenancyModels.png" imageanchor="1"><img border="0" data-original-height="541" data-original-width="828" height="418" src="https://1.bp.blogspot.com/-6ZOZxRu7e60/XotBTpWl0YI/AAAAAAAAH_s/jB7tlLy_Jiw9LTy24nd4B80a1DleydT_wCLcBGAsYHQ/s640/tenancyModels.png" width="640" /></a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Various multitenancy strategy supported by Hibernate are
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;">public enum MultiTenancyStrategy {
DISCRIMINATOR,
SCHEMA,
DATABASE,
NONE;
}
</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
DISCRIMINATOR strategy work is in progress and from the documentation of hibernate
DISCRIMINATOR strategy Correlates to the partitioned (discriminator) approach. It is an error to attempt to open a session without a tenant identifier using this strategy. </span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This strategy is not yet implemented and you can follow its progress via the HHH-6054 Jira issue. </span></div>
<div>
<br /></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://hibernate.atlassian.net/browse/HHH-6054">JIRA HHH-6054</a> is currently in OPEN Status. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here in the example following modules/projects shows the example to achieve the tenancy modelling using
SCHEMA (shared db, separate schema) strategy per tenant based approach and SCHEMA (shared db, shared schema) strategy for different tenants.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>multitenancy-op2</b> project shows the example with the multiple tenants share the same database but different schemas. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>multitenancy-op3</b> project shows the example of the hybrid approach of with the tenants are grouped against the specific database and schema.
This would give an ability to map a specific tenant with single database as well when the high load is expected for a tenant in comparison to others. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Project multitenancy-op2</b>
UseCase is where database is same (<b>dbA</b>) and every TENANT is using specific Schema i.e.
Tenant wise DB and Schema mapping. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<div>
</div>
<div>
</div>
<div>
</div>
<div>
</div>
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; width: 261px;"><colgroup><col span="3" style="width: 65pt;" width="87"></col></colgroup><tbody>
<tr height="21" style="height: 16pt;"><td class="xl64" height="21" style="background: rgb(244, 176, 132); border: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; font-weight: 700; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap; width: 65pt;" width="87">TENANTID</td><td class="xl64" style="background: rgb(244, 176, 132); border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; font-weight: 700; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap; width: 65pt;" width="87">DB </td><td class="xl64" style="background: rgb(244, 176, 132); border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; font-weight: 700; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap; width: 65pt;" width="87">SCHEMA</td></tr>
<tr height="21" style="height: 16pt;"><td class="xl63" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">TENANT1 </td><td class="xl63" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">dbA</td><td class="xl63" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">SCHEMA1</td></tr>
<tr height="21" style="height: 16pt;"><td class="xl63" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">TENANT2 </td><td class="xl63" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">dbA</td><td class="xl63" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">SCHEMA2</td></tr>
</tbody></table>
</div>
<div>
<br /></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Build and run multitenancy-op2 project</b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b>
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #cccccc;">cd multitenancy-op2
mvn clean package
mvn spring-boot:run</span>
</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Flyway integration would be creating necessary tables in database dbA </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"> Catalog creation request for TENANT1 which would insert data into table SCHEMA1.CATALOGTABLE</span><br />
<pre style="text-align: left;"><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X POST \
http://localhost:8080/catalog \
-H 'content-type: application/json' \
-H 'tenantid: TENANT1' \
-d '{"tenantId":"TENANT1","catalogName":"Catalog1","supplierId":"supplier1","source":"X-Systems"}'</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Catalog creation request for TENANT2 which would insert data into table SCHEMA2.CATALOGTABLE
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X POST \
http://localhost:8080/catalog \
-H 'content-type: application/json' \
-H 'tenantid: TENANT2' \
-d '{"tenantId":"TENANT2","catalogName":"Catalog2","supplierId":"supplier2","source":"Y-Systems"}'</span></pre>
</div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Catalog retrieval request for TENANT1</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X GET \
'http://localhost:8080/catalog?catalogId=1' \
-H 'content-type: application/json' \
-H 'tenantid: TENANT1'</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Catalog retrieval request for TENANT2
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X GET \
'http://localhost:8080/catalog?catalogId=2' \
-H 'content-type: application/json' \
-H 'tenantid: TENANT2'</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>
Project multitenancy-op3</b>
UseCase is as the hybrid approach to accommodate following cases: </span>
<br />
<div>
<ol style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Tenant(s) with higher load using the dedicated database and single schema (<b>TENANT1</b>). </span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Tenant(s) with moderate load can use same database but different schema (<b>approach in multenancy-op2 project</b>). </span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Tenant(s) with lower loads can share the DB SCHEMA (<b>TENANT2 and TENANT3</b>).
Tenant wise DB and Schema mapping.</span></li>
</ol>
</div>
<div>
</div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<div>
</div>
<div>
</div>
<div>
</div>
<div>
</div>
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; text-align: center; width: 261px;"><colgroup><col span="3" style="width: 65pt;" width="87"></col></colgroup><tbody>
<tr height="21" style="height: 16pt;"><td class="xl66" height="21" style="background: rgb(244, 176, 132); border: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; font-weight: 700; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap; width: 65pt;" width="87"><div style="text-align: center;">
TENANTID</div>
</td><td class="xl66" style="background: rgb(244, 176, 132); border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; font-weight: 700; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap; width: 65pt;" width="87">DB </td><td class="xl66" style="background: rgb(244, 176, 132); border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; font-weight: 700; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap; width: 65pt;" width="87">SCHEMA</td></tr>
<tr height="21" style="height: 16pt;"><td class="xl65" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">TENANT1 </td><td class="xl65" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">dbA</td><td class="xl65" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">SCHEMA1</td></tr>
<tr height="21" style="height: 16pt;"><td class="xl65" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">TENANT2 </td><td class="xl65" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">dbB</td><td class="xl65" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">SHARED</td></tr>
<tr height="21" style="height: 16pt;"><td class="xl65" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">TENANT3 </td><td class="xl65" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">dbB</td><td class="xl65" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">SHARED</td></tr>
</tbody></table>
</div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Build and run multitenancy-op3 project</b>
</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #cccccc;">cd multitenancy-op3
mvn clean package
mvn spring-boot:run</span></span></pre>
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #cccccc;">
</span></span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Create the following schema manually</b>
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;">create schema if not exists SHARED;
create table SHARED.catalogtable(id bigint, tenantid varchar(10), catalogname varchar(30), supplierid varchar(30), source varchar(10));
CREATE SEQUENCE SHARED.hibernate_sequence START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1;
</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b></span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></b></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>
Sample request for catalog creation</b><br />
Catalog creation request for TENANT1 which would insert data into table SCHEMA1.CATALOGTABLE in database dbA
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X POST \
http://localhost:8080/catalog \
-H 'content-type: application/json' \
-H 'tenantid: TENANT1' \
-d '{"tenantId":"TENANT1","catalogName":"Catalog1","supplierId":"supplier1","source":"X-Systems"}'</span></span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Catalog creation request for TENANT2 which would insert data into table SHARED.CATALOGTABLE in database dbB</span><br />
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X POST \
http://localhost:8080/catalog \
-H 'content-type: application/json' \
-H 'tenantid: TENANT2' \
-d '{"tenantId":"TENANT2","catalogName":"Catalog2","supplierId":"supplier2","source":"X-Systems"}'</span></span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Catalog creation request for TENANT3 which would insert data into table SHARED.CATALOGTABLE in database dbB</span><br />
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X POST \
http://localhost:8080/catalog \
-H 'content-type: application/json' \
-H 'tenantid: TENANT3' \
-d '{"tenantId":"TENANT3","catalogName":"Catalog3","supplierId":"supplier3","source":"Y-Systems"}'</span></span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Inserted records can be verified at http://localhost:8080/h2-console/ by logging into </span></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">use database <b>dbA</b> (jdbc:h2:mem:dbA) for TENANT1 in schema SCHEMA1
<a href="https://2.bp.blogspot.com/-vVep_6nxxjY/XotBXeL5OVI/AAAAAAAAH_w/KbIc6koB2Z43njUGECSAXOb83wNOlij2gCLcBGAsYHQ/s1600/SCHEMA1.png" imageanchor="1"><img border="0" data-original-height="786" data-original-width="1510" height="332" src="https://2.bp.blogspot.com/-vVep_6nxxjY/XotBXeL5OVI/AAAAAAAAH_w/KbIc6koB2Z43njUGECSAXOb83wNOlij2gCLcBGAsYHQ/s640/SCHEMA1.png" width="640" /></a> </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"> use database <b>dbB</b> (jdbc:h2:mem:dbB) for TENANT2 and TENANT3 in schema SHARED</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://4.bp.blogspot.com/-IIcqKkGJ5NI/XotBXvFNLFI/AAAAAAAAH_0/yvgJmSCTpy4wjEpGSUlnxEdRVZIvDjrqACLcBGAsYHQ/s1600/SHARED.png" imageanchor="1"><img border="0" data-original-height="773" data-original-width="1598" height="308" src="https://4.bp.blogspot.com/-IIcqKkGJ5NI/XotBXvFNLFI/AAAAAAAAH_0/yvgJmSCTpy4wjEpGSUlnxEdRVZIvDjrqACLcBGAsYHQ/s640/SHARED.png" width="640" /></a></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sample request for catalog retrieval</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X GET \
'http://localhost:8080/catalog?catalogId=1' \
-H 'content-type: application/json' \
-H 'tenantid: TENANT1'</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Catalog </span><span style="font-family: "arial" , "helvetica" , sans-serif;">retrieval</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> request for TENANT2
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X GET \
'http://localhost:8080/catalog?catalogId=1' \
-H 'content-type: application/json' \
-H 'tenantid: TENANT2'</span></span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Catalog retrieval request for TENANT3
</span><br />
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">curl -X GET \
'http://localhost:8080/catalog?catalogId=2' \
-H 'content-type: application/json' \
-H 'tenantid: TENANT3'</span></span></pre>
<pre><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Download/Refer code from <a href="https://github.com/deepaksinghvi/multitenancy">github</a> repository.</span></div>
</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-21987652987384235212018-05-13T23:34:00.000+05:302018-05-13T23:34:06.425+05:30Smart Light using Arduino<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<br />
While working on the regular projects in office, I have joined SAP Ariba IOT Group where we try out few things which can solve some problem which could be really helpful.<br />
<br />
I am also trying to use Raspberry Pi, Arduino and other platforms. <br />
<br />
Recently i have tried to build a smart light, which I want to fix it to my study table now. :-)<br />
<br />
<b>Circuit:</b><br />
I have picked the circuit from internet which explains about the connectivity of different components.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-nYgYy2JnYHU/Wvco_JUIvQI/AAAAAAAAGN0/mQPLZqP2mmkKGIQprJL6qYyx3PXCEQSXACLcBGAs/s1600/Circuit_SmartLight.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="470" data-original-width="623" height="481" src="https://1.bp.blogspot.com/-nYgYy2JnYHU/Wvco_JUIvQI/AAAAAAAAGN0/mQPLZqP2mmkKGIQprJL6qYyx3PXCEQSXACLcBGAs/s640/Circuit_SmartLight.JPG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Circuit Diagram</td></tr>
</tbody></table>
<br />
<b>Demo Video:</b><br />
Here is a short video about it.<br />
<br />
<iframe allow="autoplay; encrypted-media" allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/PszgdI30kkw" width="560"></iframe><br />
<br />
<b>Hardware Components:</b><br />
<style>
<!--table
{mso-displayed-decimal-separator:"\.";
mso-displayed-thousand-separator:"\,";}
@page
{margin:.75in .7in .75in .7in;
mso-header-margin:.3in;
mso-footer-margin:.3in;}
tr
{mso-height-source:auto;}
col
{mso-width-source:auto;}
br
{mso-data-placement:same-cell;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:12.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:Calibri, sans-serif;
mso-font-charset:0;
mso-number-format:General;
text-align:general;
vertical-align:bottom;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
.xl65
{color:black;
font-size:14.0pt;
font-family:Times, serif;
mso-font-charset:0;
border:.5pt solid windowtext;}
.xl66
{border:.5pt solid windowtext;}
-->
</style>
<br />
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; width: 379px;">
<!--StartFragment-->
<colgroup><col style="mso-width-alt: 9344; mso-width-source: userset; width: 219pt;" width="292"></col>
<col style="width: 65pt;" width="87"></col>
</colgroup><tbody>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="height: 19.0pt; width: 219pt;" width="292">Arduino
Uno</td>
<td align="right" class="xl66" style="border-left: none; width: 65pt;" width="87">1</td>
</tr>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="border-top: none; height: 19.0pt;">Relay Module</td>
<td align="right" class="xl66" style="border-left: none; border-top: none;">1</td>
</tr>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="border-top: none; height: 19.0pt;">PIR Motion
Detector Sensor Module</td>
<td align="right" class="xl66" style="border-left: none; border-top: none;">1</td>
</tr>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="border-top: none; height: 19.0pt;">Connectors/Jumper
Wires</td>
<td align="right" class="xl66" style="border-left: none; border-top: none;">6</td>
</tr>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="border-top: none; height: 19.0pt;">Power Cable</td>
<td align="right" class="xl66" style="border-left: none; border-top: none;">1</td>
</tr>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="border-top: none; height: 19.0pt;">Power adaptor
for Arduino Uno</td>
<td align="right" class="xl66" style="border-left: none; border-top: none;">1</td>
</tr>
<tr height="25" style="height: 19.0pt;">
<td class="xl65" height="25" style="border-top: none; height: 19.0pt;">Light</td>
<td align="right" class="xl66" style="border-left: none; border-top: none;">1</td>
</tr>
<!--EndFragment-->
</tbody></table>
<br />
<br />
<b>Software Components:</b><br />
<style>
<!--table
{mso-displayed-decimal-separator:"\.";
mso-displayed-thousand-separator:"\,";}
@page
{margin:.75in .7in .75in .7in;
mso-header-margin:.3in;
mso-footer-margin:.3in;}
tr
{mso-height-source:auto;}
col
{mso-width-source:auto;}
br
{mso-data-placement:same-cell;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:12.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:Calibri, sans-serif;
mso-font-charset:0;
mso-number-format:General;
text-align:general;
vertical-align:bottom;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
.xl65
{border:.5pt solid windowtext;}
.xl66
{color:black;
font-size:14.0pt;
font-family:Times, serif;
mso-font-charset:0;
border:.5pt solid windowtext;}
-->
</style>
<br />
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; width: 379px;">
<colgroup><col style="mso-width-alt: 9344; mso-width-source: userset; width: 219pt;" width="292"></col>
<col style="width: 65pt;" width="87"></col>
</colgroup><tbody>
<tr height="25" style="height: 19.0pt;">
<!--StartFragment-->
<td class="xl66" height="25" style="height: 19.0pt; width: 219pt;" width="292">Arduino
IDE</td>
<td align="right" class="xl65" style="border-left: none; width: 65pt;" width="87">1</td>
<!--EndFragment-->
</tr>
</tbody></table>
<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-v_SO1_CJaTE/Wvclo2S1mCI/AAAAAAAAGNo/7Pt9YfUb-PkJvL8zcAc4r8vqvq_uW9hcQCLcBGAs/s1600/Arduino_SmartLight.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Smart Light Code" border="0" data-original-height="630" data-original-width="505" height="640" src="https://3.bp.blogspot.com/-v_SO1_CJaTE/Wvclo2S1mCI/AAAAAAAAGNo/7Pt9YfUb-PkJvL8zcAc4r8vqvq_uW9hcQCLcBGAs/s640/Arduino_SmartLight.png" title="Smart Light" width="512" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Smart Light Code</td></tr>
</tbody></table>
<br />
<b>Code:</b><br />
<script src="https://gist.github.com/deepaksinghvi/4c1ba84ab3e8b4e50901661386ec8859.js"></script><br />
<br />
<br />
<b>Troubleshoot:</b><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="background-color: white; font-size: 12.8px;">While uploading the code there was an issue with PORT(</span></span><span style="color: #222222; font-family: arial, sans-serif; font-size: 12.8px;"> </span><span style="color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">/dev/ttyACM on ubuntu)</span><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;"> read/write permission</span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="background-color: white; font-size: 12.8px;">Following commands helped:</span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;"><br /></span></span>
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;">deepak@deepak-B570:$ ls -l /dev/ttyACM*</span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;">crw-rw---- 1 root dialout 166, 0 May 12 19:21 /dev/ttyACM0</span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;"><br /></span></span>
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;">deepak@deepak-B570:$ ls -l /dev/ttyACM*</span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;">crw-rw---- 1 root dialout 166, 0 May 12 19:21 /dev/ttyACM0</span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;"><br /></span></span>
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;">deepak@deepak-B570:$ sudo chmod a+rw /dev/ttyACM0</span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="background-color: white; font-size: 12.8px;"></span></span><br />
<span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8px;">[sudo] password for deepak: ********</span></span></div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-91853863918141917692017-01-11T00:27:00.001+05:302021-04-11T00:15:08.489+05:30Microservices for better scale<div dir="ltr" style="text-align: left;" trbidi="on"><span> </span><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-OniR9sOB2G4/YHHyLssPibI/AAAAAAAAIWk/bLlcxwD9emApIto_jJDHRiN8LyBPscDlwCLcBGAsYHQ/s2048/microservices.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1365" data-original-width="2048" src="https://1.bp.blogspot.com/-OniR9sOB2G4/YHHyLssPibI/AAAAAAAAIWk/bLlcxwD9emApIto_jJDHRiN8LyBPscDlwCLcBGAsYHQ/s320/microservices.jpeg" width="320" /></a></div><br /><div dir="ltr" style="text-align: left;" trbidi="on">
I was reading some article and which triggered the question in my mind w.r.t business services particularly "Microservices":<br />
<br />
<br />
<ul style="text-align: left;">
<li>What do I need from <a href="https://en.wikipedia.org/wiki/CAP_theorem">CAP</a> theorem, should System be AP system (consul, eureka, etc) or CP system (zookeeper, etcd, etc). How to decide about it?</li>
<li>Figure out how to run, manage, and monitor these systems at scale. How to plan for it?</li>
</ul>
<br />
<br />
Some of the points were answered once I built a on the small microservice demo.<br />
<br />
Following slide would help:<br />
<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="485" marginheight="0" marginwidth="0" scrolling="no" src="//www.slideshare.net/slideshow/embed_code/key/qj7YOF4Y4Na0Rr" style="border-width: 1px; border: 1px solid #CCC; margin-bottom: 5px; max-width: 100%;" width="595"> </iframe> <br />
<div style="margin-bottom: 5px;">
<strong> <a href="https://www.slideshare.net/Gemsbond/microservices-forscale" target="_blank" title="Microservices forscale">Microservices forscale</a> </strong> from <strong><a href="https://www.slideshare.net/Gemsbond" target="_blank">Deepak Singhvi</a></strong> </div>
<br />
<br />
Download source code from <a href="https://github.com/deepaksinghvi/microservice">github</a>.<br />
<br />
<span style="font-size: large;"><b>Eureka (Discovery Server/Service):</b></span><br />
Eureka developed by Netflix is a REST based service that was primarily used by them in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers.<br />
<br />
Eureka also comes with a Java-based client component,the Eureka Client, which makes interactions with the service much easier. The client also has a built-in load balancer that does basic round-robin load balancing.<br />
<br />
<span style="font-size: large;"><b>Zuul (Gateway/Proxy and Load Balancer):</b></span><br />
Zuul is a JVM based router and server side load balancer by Netflix. And Spring Cloud has a nice integration with an embedded Zuul proxy.<br />
There are many usages of Zuul I found following would be very helpful:<br />
<br />
<ul style="text-align: left;">
<li>Authentication</li>
<li>Dynamic Routing</li>
<li>Service Migration</li>
<li>Load Shedding</li>
<li>Security</li>
</ul>
For proxyservice have a service and enable zuul proxy <span style="color: #a71d5d; font-family: "menlo"; font-size: 12px;">@EnableZuulProxy</span> and define the routes.<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #a71d5d; -webkit-text-stroke: #a71d5d}
span.s1 {font-kerning: none}
</style>
<br />
<script src="https://gist.github.com/deepaksinghvi/a90eea988d3a83d59aee031622339444.js"></script><br />
<br />
Routing to be configured in configuration file:<br />
<script src="https://gist.github.com/deepaksinghvi/0289302c58e9e65d7d5c48503467e384.js"></script><br />
<br />
Any request which catalogservice would be routed to serviceId catalogservice which in this case is catalogservice registered with Eureka.<br />
<br />
Spring Cloud has created an embedded Zuul proxy to ease the development of a very common use case where a UI application wants to proxy calls to one or more back end services. This feature is useful for a user interface to proxy to the backend services it requires, avoiding the need to manage CORS and authentication concerns independently for all the backends.<br />
<br />
There are few pre created filters and custom filter (see <span style="background-color: #e4e4ff; font-family: Menlo; font-size: 9pt;">PreFilter.java)</span> can also be created easily.<br />
<br />
<b style="font-size: x-large;">Ribbon (Load Balancer):</b><br />
Ribbon is a client side load balancer which gives you a lot of control over the behaviour of HTTP and TCP clients. Feign already uses Ribbon<br />
<br />
<span style="font-size: large;"><b>Feign (Web Service Client):</b></span><br />
Feing offers a solution to dynamically generate clients from the interface. Additional benefit is that we can keep the signature of the service and client identical. Just declare an interface request mappings :<br />
<br />
<script src="https://gist.github.com/deepaksinghvi/ae826893254f69b505e61156f174586c.js"></script><br />
<br />
<br />
Above example would be invoking the catalogservice's (CatalogController) getItems().<br />
<br />
The Ribbon client above would discover the physical addresses for the "catalogservice" service. If application is a Eureka client then it will resolve the service in the Eureka service registry. If Eureka is not used, than configuring a list of servers in your external configuration would also help ribbon to find and load balance appropriately.<br />
<div>
<br /></div>
<br />
:-)</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0Bangalore18.312810846425442 79.1015625-11.943724153574557 37.7929685 48.569345846425442 120.4101565tag:blogger.com,1999:blog-2415001147493991407.post-35354367981105264012016-10-12T21:03:00.000+05:302016-10-13T09:57:26.181+05:30Visualisation using d3.js based Sunburst with Apache Zeppelin<div dir="ltr" style="text-align: left;" trbidi="on">
Zeppelin provides few default visual components (pie, bar, stacked, area, line chart, etc).<br />
If users want either they can add a new default component or create visualisation using AngularJS interpreter.<br />
<br />
<div class="p1">
I tried to create d3 based Sunburst for preparing a report in Apache Zeppelin.</div>
<div class="p2">
<br /></div>
<div class="p1">
It is easy and quick.</div>
<div class="p2">
<br /></div>
<div class="p1">
Apache Zeppelin display system adds additional div(s) and which creates some blank area on the screen. </div>
<div class="p2">
<br /></div>
<div class="p1">
You can experience this as there is a blank area between sunburst and breadcrumbs in the bottom.</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-GTxPtmp_Dl4/V_5WmFUwG2I/AAAAAAAABHM/kngEISLzBSQH7kcOwTASKLgxKyn_jeiTgCLcB/s1600/NSECombinedReport2.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="400" src="https://1.bp.blogspot.com/-GTxPtmp_Dl4/V_5WmFUwG2I/AAAAAAAABHM/kngEISLzBSQH7kcOwTASKLgxKyn_jeiTgCLcB/s640/NSECombinedReport2.gif" width="640" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
********
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Notebook for the above visualisation is available <a href="https://gist.github.com/deepaksinghvi/428d977e8879d6906b5baf6b3e2a5e00">here</a> which can be imported into Apache Zeppelin.<br />
This contains the AngularJS source for sunburst visual.<br />
<br />
<br />
Content which I downloaded from NSE historical data section and transformed it for demo purpose i.e. nsecombinedreport.csv can be downloaded from <a href="https://gist.github.com/deepaksinghvi/993eb2463de97cc1a9df1be7b449f654">here</a>.<br />
This report is for the various Instrument Type, Security and the amount traded for a day.</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-19012625329252660742016-05-15T20:18:00.000+05:302016-05-15T22:10:59.741+05:30Focusing on implementing govt policies using the big data tool zeppelin<div dir="ltr" style="text-align: left;" trbidi="on">
It was good to know from the goverment that it published lots of data collected over the period of time at https://data.gov.in/<br />
<br />
I picked and amenities data about the villages from https://data.gov.in/catalog/village-amenities-census-2011 to do some analysis.<br />
<br />
I believe govterment is doing sufficient analysis to find where and with what force it should use its machinery to promote its schemes.<br />
<br />
I have been doing some analysis using the Apache Spark and eco system around it. But was interested in a quick visualization, which would help to understand the data quickly. A possible use would be using R as I wanted to build the reports quickly. I explored some of the capabilities of R and Shiny App in my earlier post of <a href="http://deepaksinghviblog.blogspot.in/2014/09/cluster-analysis-using-r-with-banking.html">Custer Analysis of banking data</a>.<br />
<br />
Recently I came to know about a fantastic tool, its a web based notebook, with the in-built support for Apache-Spark, with a support of multiple langues like Scala, Python, spark sql and so on and most important that this it is opensource.<br />
<div style="text-align: center;">
<b><span style="font-size: x-large;">"Zeppelin"</span></b> </div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-DgmGLDVH4Vw/VziMlu9YFeI/AAAAAAAABAk/zI8KllwID4sRnXoEbzEPVdigibuS28xzgCLcB/s1600/zeppelinlogo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-DgmGLDVH4Vw/VziMlu9YFeI/AAAAAAAABAk/zI8KllwID4sRnXoEbzEPVdigibuS28xzgCLcB/s1600/zeppelinlogo.png" /></a></div>
I picked one of the csv from the the whole data, and which is for one of the district in Karnataka state is Gulbarga and started doing some analysis.<br />
<br />
Loading the data into the dataframe/table<br />
<script src="https://gist.github.com/deepaksinghvi/2ab5a7d51025905281d9879cea1dbdad.js"></script><br />
<br />
It is easy to accomodate spark sql also in the notebook paragraph/sections.<br />
Following is a very simple query to show the population spread in the villages of Gulbarga district.<br />
<script src="https://gist.github.com/deepaksinghvi/407d5442d3045b986e0e143cb9211f94.js"></script><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-obov5yTYmZQ/VziKMadlP8I/AAAAAAAABAI/bSaPORXa7LAbCcxLEJYAlrL0lan2qP6sACLcB/s1600/VillagesPopulationInGulbarga.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="313" src="https://2.bp.blogspot.com/-obov5yTYmZQ/VziKMadlP8I/AAAAAAAABAI/bSaPORXa7LAbCcxLEJYAlrL0lan2qP6sACLcB/s640/VillagesPopulationInGulbarga.png" width="640" /></a></div>
<br />
Goverment make policies and spend money on that, and find the effectiveness of it based on the result. We can use the collected data to understand where should be the maximum penetration of the schemes, i.e. find the villages which needs the goverment schemes most. One of the example where goverment can initiates its policies to reduce the gap of male-female ratio, we can understand from the data available, where should be the more focus.<br />
<br />
<script src="https://gist.github.com/deepaksinghvi/5b1ed75b2719bea9c5ac10ea14ab0535.js"></script><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-EByxSAjhDTM/VziKrO8A9MI/AAAAAAAABAM/3JpGaO8EmTwZiU6yxyt94GigGZKbHckLgCLcB/s1600/MaleFemaleRatio.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="340" src="https://3.bp.blogspot.com/-EByxSAjhDTM/VziKrO8A9MI/AAAAAAAABAM/3JpGaO8EmTwZiU6yxyt94GigGZKbHckLgCLcB/s640/MaleFemaleRatio.png" width="640" /></a></div>
<br />
<br />
Changed the minbenchmark to 80% and same got updated on the fly<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-x7Qq6Q35b4s/VziLVYeu5zI/AAAAAAAABAU/m817Iae6sYY-ggMMjdJrvNPlZhETQ5ETQCLcB/s1600/MaleFemaleRatio_80.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="326" src="https://4.bp.blogspot.com/-x7Qq6Q35b4s/VziLVYeu5zI/AAAAAAAABAU/m817Iae6sYY-ggMMjdJrvNPlZhETQ5ETQCLcB/s640/MaleFemaleRatio_80.png" width="640" /></a></div>
<br />
<br />
I stated to analyse this data to check for the education facilities in the villages which is in progress, would be publishing that information in later posts.<br />
<br />
Installation details:<br />
a) Zeppelin was deployed on Ubuntu VirtualBox with Windows as host.<br />
b) Set your java home (1.7) before starting Zeppelin.<br />
c) To start execute 'zeppelin-daemon.sh start' in the ZEPPELIN_HOME\bin<br />
<br />
<br />
<br />
<br /></div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-19664123762802982062014-09-15T00:02:00.000+05:302014-09-15T00:34:01.452+05:30Cluster Analysis using R with banking customer balance distribution<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-size: 12px; line-height: 18px;">Once a wise man </span><span style="font-size: 12px; line-height: 18px;">(Yachine Phuneli)</span><span style="font-size: 12px; line-height: 18px;"> taught me while teaching me about java, semiconductor and wafer fabrication that how to teach/explain.</span></span></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px;">First tell to your audience/readers what are going to to tell.</span></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px;">Than tell the thing which you planned to tell.</span></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px;">And than tell what you just told.</span></div>
<div style="text-align: left;">
<span style="font-family: Arial, sans-serif;"><span style="font-size: 12px; line-height: 18px;"><br /></span></span></div>
<span style="color: #990000;">I am going to describe how to do Cluster Analysis using R</span><br />
<div>
<br />
<div>
<span style="font-family: Arial, sans-serif; font-size: 12pt; line-height: 18.3999996185303px;">Cluster Analysis</span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster analysis has a vital role in numerous fields we are going
to see it in the banking business to segment customers into small groups that
can later be targeted for future marketing activities.<br />
In machine learning and data mining it is used to efficiently find nearest neighbours
and in summarization.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster analysis aims to group
data objects based on the information that is available that describes the
objects and their relationships. The main goal is to group similar objects
together, and the greater the similarity within a group the better and the
greater the difference between group the more diverse the clustering. A
clustering is an entire collection of clusters; a cluster on the other hand is
just one part of the entire picture. There are different types of clusters and
also different types of clustering.<o:p></o:p></span></div>
<h2>
<span style="font-family: Arial, sans-serif; font-size: 12pt; line-height: 115%;">Types of Clustering Algorithms<o:p></o:p></span></h2>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 36.0pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">1.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><b>Partitioning-based clustering:</b> <span style="font-family: Arial, sans-serif; font-size: 9pt;">are
algorithms that determine all the clusters at once in most cases.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 72.0pt; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 72.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: 'Courier New'; font-size: 10pt;">o<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">K-means
clustering<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 72.0pt; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 72.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: 'Courier New'; font-size: 10pt;">o<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">K-medoids
clustering<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 72.0pt; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 72.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: 'Courier New'; font-size: 10pt;">o<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">EM
(expectation maximization) clustering<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 36.0pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">2.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><b>Hierarchical clustering:</b><span class="apple-converted-space"><span style="font-family: Arial, sans-serif; font-size: 9pt;"> </span></span><span style="font-family: Arial, sans-serif; font-size: 9pt;">these algorithms find successive clusters using previously
established ones.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 72.0pt; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 72.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: 'Courier New'; font-size: 10pt;">o<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Divisive
clustering is a top down approach.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-left: 72.0pt; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list 72.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: 'Courier New'; font-size: 10pt;">o<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Agglomerative
clustering is a bottom up approach.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">With the help of data mining methods, such as clustering
algorithm, it is possible to discover the key characteristics from the bank’s
data and possibly use those characteristics for future prediction also.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">According to the facts established based on the information
released by banks, attracting new customer costs five to six times more than
customer retention. Retaining existing customers is the best core marketing
strategy to become profitable in the very competitive banking industry. In
order to maximise the profit, how to retain the existing customers has become a
subject to be urgently solving for banks.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;">Use
clustering to produce an initial working hypothesis, refine this hypothesis,
then use prediction to generalize the refined hypothesis to data and evaluate
how well it performs.</span><br />
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<br />
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">For this study
following was the Bank customer data for the Balance distribution.<o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-sszNyVfsMAE/VBXXYyqULOI/AAAAAAAAAws/CFPlA4nSils/s1600/bankData.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-sszNyVfsMAE/VBXXYyqULOI/AAAAAAAAAws/CFPlA4nSils/s1600/bankData.png" height="156" width="640" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Before doing the cluster analysis above data was transformed based
on the type or range of values. This would help us to do the scaling and find
number of cluster quickly. We can keep the data as it is but I opted to find
clustering based on the simplified data.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Balance
conversion based on range:<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">=IF(AND(Sheet2!I2>=0,Sheet2!I2<=1500),"1",IF(AND(Sheet2!I2>1500,Sheet2!I2<=3000),"2",IF(AND(Sheet2!I2>3000,Sheet2!I2<6000 -=""> If Balance between 0 to 1500 : 1, If Balance between 1501 to 3000:2, If
Balance between 3000 to 6000:3, Greater than 6000 :4<o:p></o:p><!--6000--><!--6000--><!--6000--><!--6000--><!--6000--></6000></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Age
conversion based on the range:<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">=IF(AND(Sheet2!E2>=20,Sheet2!E2<=35),"1",IF(AND(Sheet2!E2>35,Sheet2!E2<=45),"2",IF(AND(Sheet2!E2>45,Sheet2!E2<=60),"3","4")))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Customer
type conversion based on type::<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">=IF(Sheet2!C6="Enterprise","1","2")<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Product
conversion ::<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">=IF(Sheet2!C6="SavingAccount","1","2") -> 1 if SavingAccount, 2 if CurrentAccount<o:p></o:p></span></div>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;">So
we got the data transformed in the following way:</span><span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><!--[endif]--></span><br />
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-Gpc_lsFYgf8/VBXXqxqggbI/AAAAAAAAAw0/1Ro_xn8SPFA/s1600/transformedBankData.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-Gpc_lsFYgf8/VBXXqxqggbI/AAAAAAAAAw0/1Ro_xn8SPFA/s1600/transformedBankData.png" height="252" width="640" /></a></div>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 115%;"><br /></span>
<br />
<div class="MsoListParagraph" style="mso-add-space: auto; mso-line-height-alt: 13.5pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-size: 18.0pt;"> </span><span style="font-family: Arial, sans-serif; font-size: 16px; line-height: 18.3999996185303px; text-indent: 0px;">Find No Of Cluster</span></div>
<br />
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">1) Load csv data from file:<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-rf7qEAglsOE/VBXX9zhQgwI/AAAAAAAAAw8/5wuh1g2ABuw/s1600/findnoofcluster_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-rf7qEAglsOE/VBXX9zhQgwI/AAAAAAAAAw8/5wuh1g2ABuw/s1600/findnoofcluster_1.png" height="338" width="640" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-ylrUK9SHMsM/VBXYIuZQ39I/AAAAAAAAAxE/TMKt6l0YWh8/s1600/findnoofcluster_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-ylrUK9SHMsM/VBXYIuZQ39I/AAAAAAAAAxE/TMKt6l0YWh8/s1600/findnoofcluster_2.png" height="328" width="640" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt; text-indent: -18pt;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt; text-indent: -18pt;"> 2)<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt; text-indent: -18pt;">Once data is loaded, per form the scaling of data by
executing </span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">bDataScale <- ankcustomerdata="" o:p="" scale=""><!-----><!-----><!-----><!-----><!-----></-></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-GnTPrezO3EQ/VBXYZyqPEoI/AAAAAAAAAxM/F6loxTEDqZg/s1600/findnoofcluster_3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-GnTPrezO3EQ/VBXYZyqPEoI/AAAAAAAAAxM/F6loxTEDqZg/s1600/findnoofcluster_3.png" height="326" width="640" /></a></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt; text-indent: -18pt;"><br /></span>
<br />
<div style="text-align: left;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt; text-indent: -18pt;"> 3)<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt; text-indent: -18pt;">Loading NBClust library and find the number of clusters for the
data available in bdata,</span></div>
</div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">library(NbClust)<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">nc <- bdatascale="" max.nc="15," method="kmeans" min.nc="2," nbclust="" o:p=""><!-----><!-----><!-----><!-----><!-----></-></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Above
command takes a while to calculate.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Following is the output of
NbClust function<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">The Hubert index is a graphical
method of determining the number of clusters.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">In the plot of Hubert index, we
seek a significant knee that corresponds to a significant increase of the value
of the measure i.e the significant peak in Hubert index second differences plot.
<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">The D index is a graphical method
of determining the number of clusters. In the plot of D index, we seek a
significant knee (the significant peak in Dindex second differences plot) that
corresponds to a significant increase of the value of the measure. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">All 1000 observations were used. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">*******************************************************************
<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* Among all indices:
<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 4 proposed 2 as the best number
of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 9 proposed 3 as the best number
of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 1 proposed 7 as the best number
of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 1 proposed 8 as the best number
of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 2 proposed 9 as the best number
of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 2 proposed 11 as the best
number of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 2 proposed 12 as the best
number of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">* 2 proposed 15 as the best
number of clusters <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"> ***** Conclusion ***** <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"> * According to the majority rule, the best
number of clusters is 3 <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-f0ZkUrWfjUQ/VBXY0PVR5ZI/AAAAAAAAAxU/Cqjq9ySJONg/s1600/findnoofcluster_4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-f0ZkUrWfjUQ/VBXY0PVR5ZI/AAAAAAAAAxU/Cqjq9ySJONg/s1600/findnoofcluster_4.png" height="340" width="640" /></a></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<div style="text-align: left;">
<span style="line-height: 13.5pt; text-indent: -18pt;"><br /></span></div>
<div style="text-align: left;">
<span style="line-height: 13.5pt; text-indent: -18pt;">4)</span><span style="font-size: 7pt; line-height: normal; text-indent: -18pt;">
</span><span style="line-height: 13.5pt; text-indent: -18pt;">Plot the chart with the number of cluster we have obtained.</span></div>
</div>
<div class="MsoListParagraph" style="line-height: 13.5pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<o:p></o:p></div>
<div class="MsoListParagraph" style="line-height: 13.5pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-leGO9gx6eTU/VBXZGGHhLHI/AAAAAAAAAxc/ObfC3JxHzto/s1600/findnoofcluster_5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-leGO9gx6eTU/VBXZGGHhLHI/AAAAAAAAAxc/ObfC3JxHzto/s1600/findnoofcluster_5.png" /></a></div>
<div class="MsoListParagraph" style="line-height: 13.5pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<br /></div>
<div class="MsoListParagraph" style="line-height: 13.5pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<br /></div>
<div class="MsoListParagraph" style="line-height: 13.5pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<br /></div>
<div class="MsoListParagraph" style="mso-add-space: auto; mso-line-height-alt: 13.5pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-size: 18.0pt; mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"> </span><span style="font-size: 18.0pt;">Calculate K-means cluster<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"> Now to find the cluster based on Quarterly average balance, </span><span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt;">bdata loaded from .csv file is</span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-tNr1XuYJEyQ/VBXZblm-yEI/AAAAAAAAAxk/w2P-mFRW46o/s1600/kMeansCluster_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-tNr1XuYJEyQ/VBXZblm-yEI/AAAAAAAAAxk/w2P-mFRW46o/s1600/kMeansCluster_1.png" height="166" width="640" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-8N_TzXDXAxY/VBXZmEujbrI/AAAAAAAAAxs/OKi_zmZ18xQ/s1600/kMeansCluster_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-8N_TzXDXAxY/VBXZmEujbrI/AAAAAAAAAxs/OKi_zmZ18xQ/s1600/kMeansCluster_2.png" height="338" width="640" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt; line-height: 13.5pt;"><br /></span></div>
<div class="MsoNormalCxSpFirst">
<br /></div>
<div class="MsoNormalCxSpLast" style="margin-bottom: .0001pt; margin-bottom: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Here I want to consider only
AvgBalQ1, AvgBalQ2, AvgBalQ3, AvgBalQ4 for kmeans cluster.<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 54.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;"> test <-bdata c="" o:p=""><!---bdata--><!---bdata--><!---bdata--><!---bdata--><!---bdata--></-bdata></span></div>
<div class="MsoNormal" style="margin-bottom: .0001pt; margin-bottom: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Scale data<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 54.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">scaledata
<-scale o:p="" test=""><!---scale--><!---scale--><!---scale--><!---scale--><!---scale--></-scale></span></div>
<div class="MsoNormal" style="margin-bottom: .0001pt; margin-bottom: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Set seet so that every time we calculate the kmeans it would be
consistent<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 54.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;"> set.seed(1234)<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: .0001pt; margin-bottom: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Find the number of cluster/center as 3,<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 54.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;"> km <- centers="3," iter.max="500)<o:p" kmeans="" nstart="10," scaledata=""><!-----><!-----><!-----><!-----><!-----></-></span></div>
<div class="MsoNormal" style="margin-bottom: .0001pt; margin-bottom: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Check the cluster size data, 401 data in cluster 1, 310 is cluster
2 and 289 in cluster 3.<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 54.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 9pt;">Ø<span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;"> km$size<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoNormalCxSpFirst" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 18.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; text-indent: 36.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">[1] 401 310 289<o:p></o:p></span></div>
<div class="MsoNormalCxSpFirst" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 18.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; text-indent: 36.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="MsoNormalCxSpFirst" style="margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 18.0pt; margin-right: 0cm; margin-top: 0cm; mso-add-space: auto; mso-line-height-alt: 0pt; text-indent: 36.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Let’s have a look of the function kmeans<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">kmeans(x, centers, iter.max = 10,
nstart = 1, algorithm = c("Hartigan-Wong", "Lloyd",
"Forgy", "MacQueen"))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;"><u>Input to kmeans function</u><o:p></o:p></span></b></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">x:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> A numeric matrix of data, or an object that can be coerced
to such a matrix (such as a numeric vector or a data frame with all numeric
columns).<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">centers:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> Either the number of clusters or a set of initial (distinct)
cluster centers. If a number, a random set of (distinct) rows in x is chosen as
the initial centers.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">iter.max:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> The maximum number of iterations allowed.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">nstart:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> If centers is a number, nstart gives the
number of random sets that should be chosen.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">algorithm:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> The algorithm to be used. It should be one
of "Hartigan-Wong", "Lloyd", "Forgy" or
"MacQueen". If no algorithm is specified, the algorithm of Hartigan
and Wong is used by default.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><b><u>Result returned from kmeans function call</u><o:p></o:p></b></span></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">cluster:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> A vector of integers indicating the cluster to which each
point is allocated.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">centers:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> A matrix of cluster centers.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">whithnss:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> The within-cluster sum of squares for each cluster.<o:p></o:p></span></div>
<div class="MsoNormalCxSpFirst" style="margin: 0cm 0cm 0.0001pt 18pt; text-indent: 36pt;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">size:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> The number of points in each cluster.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-dYQaMsM70E8/VBXauwzI89I/AAAAAAAAAx4/L-Jq6_JSLx4/s1600/WhyAndHowCluster.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-dYQaMsM70E8/VBXauwzI89I/AAAAAAAAAx4/L-Jq6_JSLx4/s1600/WhyAndHowCluster.png" /></a></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: Quarter
wise Product data distribution)<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Now if bank wants take some necessary actions on the data by
grouping the customer base and start a product or campaign for them, For e.g. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a)<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">If we see the above data distribution where bank wants to suggest
the high net worth/higher account balance customer from the saving account to opt
for other product, or <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b)<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">If bank wants to offer those customers or a group of customers
(cluster) additional services without charges.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Partitioning of data and making cluster/group would be helpful to
take necessary actions for every cluster. It would be easy to group customers
in cluster and then plan business activities on respective clusters
accordingly.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">K-means</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> clustering is the most popular partitioning method. It
requires the analyst to specify the number of clusters to extract. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"> So if we group data in 3
clusters (see the next chapter to see how to do we get 3 as a number of cluster)</span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-D3XSbFLowBI/VBXa_EZLABI/AAAAAAAAAyA/hFg6izFPdPI/s1600/WhyAndHowCluster_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-D3XSbFLowBI/VBXa_EZLABI/AAAAAAAAAyA/hFg6izFPdPI/s1600/WhyAndHowCluster_2.png" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: Quarterly
Average Balance with 3 cluster data distribution)<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">If we see these four
graphs for every quarter, Cluster 3 (in blue color) has some customer whose
balance is greater than customer in other clusters. We can mark these clusters
back to the original data and find those specific customers from the cluster to
take appropriate action. <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Kmeans clustering with 3 clusters of sizes 401, 310, 289 <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster means:<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"> AvgBalQ1 AvgBalQ2 AvgBalQ3 AvgBalQ4<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 1 4019 2189
5059 5497<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 2 2808 6413
3630 4044<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 3 5611 6527
7222 6638<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Percentage of within cluster variance accounted for by each
cluster: <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 1: <b>42.22%, </b> Cluster 2: <b>26.07%</b> Cluster 3: <b>31.72%</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><b><br /></b></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Note:<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">For every measure or non-factor or numeric value based parameter
we choose these graph would be keep
changing. Because their cluster means would be changing and so their
variance.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">For e.g. If we try to obtain data with 3 cluster but by including
age also in the data, than our cluster size, mean and cluster data percentage
changes.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Kmeans clustering with 3 clusters of sizes 353, 366, 281 <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster means:<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: 36.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">
Age AvgBalQ1 AvgBalQ2 AvgBalQ3 AvgBalQ4<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 1 31.5
3658
3858 5972 4869<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 2 59.8 3366 4640 2972
4882<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 3 57.1 5624 6023 7279
6659<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Percentage of within cluster variance accounted for by each
cluster: <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Cluster 1: <b>34.21%</b>, Cluster
2: <b>34.84%</b>, Cluster 3: <b>30.95%</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><b><br /></b></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><b><br /></b></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
Now lets do some more analysis with the with the distribution of the data.</div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Data Interpretation: Understand the data based on the different
combination of factors/dimensions and measures.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Factors/Dimensions:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> A
dimension is a broad grouping of related data about a major aspect of your
business. For example, you have a dimension called Products.<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: 12.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">Measure:</span></b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> A measure
is a performance indicator that is quantifiable and used to determine how well
a business is operating. For example, useful measures may be Average yearly
balance.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">In the following figures, I tried to figure out only two
interpretation (there could be more) and two possible actions which can be
taken.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">Note: I not sure which city exists in which state in US. I have
just generated cities and states using talend (how to generate data using talend, check in the <a href="http://deepaksinghviblog.blogspot.in/2014/08/talend-big-data-integration-with-hadoop.html">previous post</a>) for the purpose of this study, as
these cities are states were available in RowGenerator for generating records and
moved some of them to make data look better.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-PZvXhsAfBgE/VBXbhVKu0bI/AAAAAAAAAyM/GcpqnWLIzTg/s1600/DataDistribution_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-PZvXhsAfBgE/VBXbhVKu0bI/AAAAAAAAAyM/GcpqnWLIzTg/s1600/DataDistribution_1.png" height="548" width="640" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: Product
wise Gender data distribution)<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;">If we try to analyse the above graph, it suggest that there are
few female customer for a bank who has current account as product. Which are
interesting and a bank can interrupt it in two ways:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">I.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Interpretation:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Either these customers are unware about which product they should
have and they are paying unnecessary charges for CA, rather than going for SA.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Or, they are the some entrepreneur or running small business. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">II.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Actions:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Customer can be notified by the bank executive about the possible
change in their product from CA to SA and win the loyalty of customer by
showing them bank care for their customer’s money.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Marketing team target these customers with their specific needs or
keep these customer’s in mind for the future product which would meet for these
specific female customer who has CurrentAccount and are entrepreneur.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-0LblskOnd0Q/VBXbu-8-3vI/AAAAAAAAAyU/czEcQuGrJcs/s1600/DataDistribution_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-0LblskOnd0Q/VBXbu-8-3vI/AAAAAAAAAyU/czEcQuGrJcs/s1600/DataDistribution_2.png" /></a></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: Age
wise Gender data distribution)<o:p></o:p></span></b></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">I.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Data Interpretation:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Male customers are more rather than female or enterprise
customers.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Enterprise customer’s mainly lying in the range of age from 38-67.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">II.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Possible Actions:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Either products for Enterprises and Female customer are not very
effective, so few new product which is more suitable to them can be launched
with some campaign, etc.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 54.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Start-up(Enterprise) does not have any enough accounts with the
bank as the most of the customers are in the range of 40-70 and few of them are
from 20-40. If an organization/startup can open account with the bank, there
would be chances to get more salary accounts also from the same enterprise
customer.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-6vkzHaZQ4Vc/VBXb9KabOaI/AAAAAAAAAyc/CyysqV7oIR0/s1600/DataDistribution_3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-6vkzHaZQ4Vc/VBXb9KabOaI/AAAAAAAAAyc/CyysqV7oIR0/s1600/DataDistribution_3.png" height="552" width="640" /></a></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: City
wise Gender data distribution)<o:p></o:p></span></b></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">I.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Data Interpretation<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Albany city has more enterprise customers than all other cities.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Atlanta city has more personal customer (male + female)<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">II.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Possible Actions:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Any campaigns which are targeted for personal customers should
include cities like Atlanta.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Any campaigns which are targeted for enterprise customer should
include cities like Albany.<o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-_47SDPTPHQ8/VBXcIzCGsDI/AAAAAAAAAyk/WkQFVIhQ_kc/s1600/DataDistribution_4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-_47SDPTPHQ8/VBXcIzCGsDI/AAAAAAAAAyk/WkQFVIhQ_kc/s1600/DataDistribution_4.png" /></a></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: City
wise Gender and State wise Gender data distribution)<o:p></o:p></span></b></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-Tqwx7hPBsC0/VBXcYmvz3nI/AAAAAAAAAys/zwDD_ndvoVQ/s1600/DataDistribution_5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-Tqwx7hPBsC0/VBXcYmvz3nI/AAAAAAAAAys/zwDD_ndvoVQ/s1600/DataDistribution_5.png" /></a></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></b></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: Per Quarter
Average Balance per Gender)<o:p></o:p></span></b></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">I.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Data Interpretation<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Q3 and Q4 average balance increases in comparison to Q1 and Q2.
See the movement of boxplots not only the dots, in all quarters.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Q1 and Q2 Enterprise customer are more towards lesser balance in
comparison to Q3 and Q4. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">II.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Possible Actions:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">If these are the regular trends for every year, that shows that cash
flow for these customer increases in
last two quarters than it would be more appropriate time for having campaigns.
Bank would be able to get better return on their investment on those campaigns.<o:p></o:p></span></div>
<div align="center" class="MsoNormalCxSpFirst" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span class="MsoSubtleEmphasis">ROA = Margin * Asset Velocity<o:p></o:p></span></div>
<div align="center" class="MsoNormalCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span class="MsoSubtleEmphasis">Asset velocity = Sales / Assets<o:p></o:p></span></div>
<div align="center" class="MsoNormalCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span class="MsoSubtleEmphasis">More sales with more competitive product.<o:p></o:p></span></div>
<div align="center" class="MsoNormalCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span class="MsoSubtleEmphasis">More sales mean more asset velocity.<o:p></o:p></span></div>
<div align="center" class="MsoNormalCxSpMiddle" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span class="MsoSubtleEmphasis">More asset velocity means more return on
assets.<o:p></o:p></span></div>
<div align="center" class="MsoNormalCxSpLast" style="line-height: 13.5pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<span class="MsoSubtleEmphasis">More ROA is more profit.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Bank can offer few additional facilities for the customer who are
not using overdraft or limit facility. Or based on the economic environment (linking<i> of external data with bank dataset</i>) in
the country if organisations are looking for funds for capital expenditure,
bank can offer products accordingly.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-Red0Zvod7Tc/VBXcnT867FI/AAAAAAAAAy0/_kBpI4KfZWY/s1600/DataDistribution_6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-Red0Zvod7Tc/VBXcnT867FI/AAAAAAAAAy0/_kBpI4KfZWY/s1600/DataDistribution_6.png" /></a></div>
<div class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;"> (Fig1: Gender wise Product data distribution) (Fig2: Yearly Average Balance per Product)<o:p></o:p></span></b></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">I.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Data Interpretation<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">There are more female current account customers than male. (Fig 1)<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">There are few customers who have saving account but their average
balance is more than the normal data population. (Fig 2)<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">II.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Possible Actions:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Create new product or launch campaign or correct current products
to increase the male customer base with current account.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Customer can be suggested to move their funds from savings
accounts to fixed deposit this would help bank to win the loyalty of customer
by showing them bank care for their customer’s money.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-p1O7Qxcb_rM/VBXc3c_sQzI/AAAAAAAAAy8/6Xmvd8tO4NE/s1600/DataDistribution_7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-p1O7Qxcb_rM/VBXc3c_sQzI/AAAAAAAAAy8/6Xmvd8tO4NE/s1600/DataDistribution_7.png" height="404" width="640" /></a></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<div align="center" class="MsoNormal" style="line-height: 13.5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-align: center;">
<b><span style="font-family: Arial, sans-serif; font-size: 9pt;">(Fig: Quarter
wise State data distribution)<o:p></o:p></span></b></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">I.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Data Interpretation<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Alaska and Louisiana has more customer base. Florida has the least
customer base.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 18.0pt; mso-add-space: auto; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -18.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">II.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Possible Actions:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">a.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Specific marketing campaigns should be planned to get customer
base in the states where number of customer are not enough.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">b.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">If banking operations (operational efficiency) are the problem for
the lower number of customer which can be identified by <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="line-height: 13.5pt; margin-left: 108.0pt; mso-add-space: auto; mso-list: l0 level3 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -9.0pt; text-indent: -108.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span>i.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">Checking
the number of accounts/customer trend every year and see if customer base has a
diminishing trend in every year. Find the exact problem and take corrective
actions for it, one of the example for reducing customer base is the
operational efficiency. So perform check about operational efficiency of every
branch in Florida and other lower customer base states.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; text-indent: -18.0pt;">
</div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 108.0pt; mso-add-space: auto; mso-list: l0 level3 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -9.0pt; text-indent: -108.0pt;">
<!--[if !supportLists]--><span style="font-family: Arial, sans-serif; font-size: 9pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;">
</span>ii.<span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><!--[endif]--><span style="font-family: Arial, sans-serif; font-size: 9pt;">There was
not enough marketing or campaign done as these were not the area of focus in
the previous years.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: 13.5pt; margin-left: 108.0pt; mso-add-space: auto; mso-list: l0 level3 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-text-indent-alt: -9.0pt; text-indent: -108.0pt;">
<span style="font-family: Arial, sans-serif; font-size: 9pt;"><br /></span></div>
<span style="color: #990000;">Cluster analysis is one of the important technique to analyse your data, which is not easy specially if you are not statistician/mathematician. R is wonderful tool to do all the analytics work with a ease. I have used R, RStudio and ShinyApp (Web-based framework for R for more practical visualization and statistical analysis )</span></div>
<div>
<span style="color: #990000;"><br /></span>
<br />
<div>
Thanks to my friend and colleague "Alex" at Misys to introduce me to this wonderful tool and technology The "R<span style="color: #333333; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif;"><span style="background-color: white; font-size: 15px; line-height: 20.7900009155273px;">"</span></span>.<br />
<br />
<br /></div>
</div>
</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com1tag:blogger.com,1999:blog-2415001147493991407.post-29250057874242612652014-08-17T15:29:00.001+05:302014-08-17T15:29:26.584+05:30Talend Big Data Integration with Hadoop<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
Hadoop can be downloaded from the Apache Hadoop website at hadoop.apache.org. This would include core modules like Hadoop Common, Hadoop Distributed File System (HDFS), Hadoop YARN, and Hadoop MapReduce. Additional Hadoop-related projects like Hive, Pig, Hbase and many more can be downloaded from their respective Apache websites. But setting up and doing all this work is not easy.<br />
<br />
Once this is done, doing the data-integration is another challenge.<br />
Writing each and everything for the data-integration directly with the pig or hive or other script is bit difficult route to do the required work.<br />
<br />
Instead of worrying and doing too much, I think easier way would be to use the sandbox provided by any of hadoop vendor and use the data integration platform by talend. Good thing is this that both hadoop installation and talend big data integration studio software is free and available under apache license :-)<br />
<br />
Some of the hadoop vendors are HortonWorks, Cloudera, MapR, etc.<br />
<br />
<br />
When I first tried the HortonWorks with talend big data integration platform, it made things very easy.<br />
Of-course for enterprise or cloud there is license. But as I mentioned eaerlier, these are available under apache license and free.<br />
<br />
So thought of describing how this can be done. In the later post I can explain how talend and hortonworks can be used for their different features. So this post is kind of overview and later post can be in detail for different features.<br />
<br />
So what all we need.<br />
1) Virtual machine where hadoop can be installed.<br />
2) Hadoop applicance to be installed on virtual machine, i.e. HortonWorks sandbox.<br />
3) Talend Open Studio for Big Data<br />
4) Create a job for data load on HDFS.<br />
<br />
<b><span style="color: #e06666;">1) <u>Virtual Machine</u>:</span></b><br />
I have use oracle virtual box from https://www.virtualbox.org. There was an issue with 4.3 latest version so i used 4.3.12 from the link: https://www.virtualbox.org/wiki/Download_Old_Builds_4_3<br />
One you are finished with installation of virtual box, proceed for next step.<br />
<br />
<b><span style="color: #e06666;">2) <u>Hadoop applicance to be installed on virtual machine, i.e. HortonWorks sandbox</u>:</span></b><br />
a<b>)</b> <b>Download:</b><br />
There are multiple sandbox available from HortonWorks for virtualbox, vmvare, Hyper-v. As we have<br />
installed virtualbox, download that one. It can be donwloaded from the following link:<br />
http://hortonworks.com/products/hortonworks-sandbox/#install<br />
<br />
<b>b) Setup</b>:<br />
Start virtualbox, Go to file menu and click on Import Alliance.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-dqVE_-D0QEc/U_BgqBOlJnI/AAAAAAAAApU/37kV84iju_o/s1600/importAlliance.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-dqVE_-D0QEc/U_BgqBOlJnI/AAAAAAAAApU/37kV84iju_o/s1600/importAlliance.png" height="233" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Fig: Importing of HortonWorks hadoop appliance-1.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Select your appliance(.ova) file and click on next. You would see the import virtual appliance wizard,<br />
select the appropriate memory and other parameters based on your need. You can set the network<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-xpOTraMmL08/U_BgtEo1-zI/AAAAAAAAApc/XLkIKKkFWyQ/s1600/ImportAlliancePart2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-xpOTraMmL08/U_BgtEo1-zI/AAAAAAAAApc/XLkIKKkFWyQ/s1600/ImportAlliancePart2.png" height="234" width="320" /></a></div>
<div style="text-align: center;">
Fig: Importing of HortonWorks hadoop appliance-2.</div>
<div style="text-align: center;">
<br /></div>
Change the network settings to get it's own IP for virtual box. By default it would be NAT change it to Bridge Adapter.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-xPCjJ2y0q0M/U_BoJihptWI/AAAAAAAAAps/16NhywApWLI/s1600/SandboxNetworkSettings.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-xPCjJ2y0q0M/U_BoJihptWI/AAAAAAAAAps/16NhywApWLI/s1600/SandboxNetworkSettings.png" height="185" width="320" /></a></div>
<div style="text-align: center;">
Fig: Setting of the Network Adapter.</div>
<b>c) Verify</b> <br />
<br />
<ul style="text-align: left;">
<li> You can verify your installation and find the IP of VM running hadoop.</li>
<li> Click on the HortonWorks sandbox and click on start button. It will take some time to boot start all services of hadoop, pig, hive, etc and will prompt for authetication.</li>
<li> Username is "root" and password is "hadoop".</li>
<li> On the command prompt type ifconfig (as its linux) and note down the ip which would be used in talend studio during the HDFSConnection creation.</li>
<li> You can use the IP with Port 8000 (default) to check about the successful installation and running of services.</li>
</ul>
<br />
<br />
You can browse the various services at : http://xxx.yyy.xxx.zzz:8000/<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-8TxxowziVKo/U_BplD0pOmI/AAAAAAAAAp4/ean1ksBxngo/s1600/HadoopRunning.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-8TxxowziVKo/U_BplD0pOmI/AAAAAAAAAp4/ean1ksBxngo/s1600/HadoopRunning.png" height="353" width="640" /></a></div>
<div style="text-align: center;">
Fig: Accessing hadoop services using the browser from remote machine.</div>
<br />
<br />
<br />
You can execute some script or browse the older scripts from the pig menu.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-xmO8zsevW7M/U_BsFd5I1uI/AAAAAAAAAqE/hhRM8MJQFw8/s1600/PigBrowser.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="http://2.bp.blogspot.com/-xmO8zsevW7M/U_BsFd5I1uI/AAAAAAAAAqE/hhRM8MJQFw8/s1600/PigBrowser.png" height="353" width="640" /></a></div>
<span style="text-align: center;"> Fig: Accessing and running pig script from remote machine.</span><br />
<span style="text-align: center;"><br /></span>
<br />
<br />
<b><span style="color: #e06666;">3) <u>Talend open studio</u>:</span></b><br />
<span style="color: #4c1130;"><b> </b>You can download the talend open studio for big data (not the data integration ) from the following link:</span><br />
http://www.talend.com/download?qt-download_landing=0#quicktabs-download_landing<br />
<br />
If you want you can download Talend Big Data Sandbox for any of the hadoop provider but as i wanted to keep these two on separate machine which would be the ideal case.<br />
It would be an .exe file if you are installation on windows, execute it and select the installation location and you are done with the talend open studio installation.<br />
<br />
<span style="color: lime;"> </span><span style="color: #e06666;">4<b>) <u>Create a job for data load on HDFS</u>:</b></span><br />
<b style="color: #4c1130;"> </b>What we want to do here is to create a job which generate few records and write it to HDFS on the hadoop we have installed in the step 2. Createing and running the job is happening on my local box and hadoop is running on some remote machine.<br />
<br />
At the end it would look like following and execution of the job would create a file of 100 records in HDFS of hadoop.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-wj9I0x1tWbo/U_BvE2jyWAI/AAAAAAAAAqQ/lRMOpNznO5c/s1600/TalenHDFSHadoopJob.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-wj9I0x1tWbo/U_BvE2jyWAI/AAAAAAAAAqQ/lRMOpNznO5c/s1600/TalenHDFSHadoopJob.png" height="255" width="400" /></a></div>
<br />
a) Start talend by starting TOS_BD-win-x86_64.exe<br />
b) Create a new project and open it.<br />
c) Design a job: Following steps would make job designing complete.<br />
<div style="text-align: left;">
</div>
<div style="text-align: left;">
i)) Create a new job by right click on the Repository->Job Design</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-UFAvvVvNMPU/U_Bv867v3II/AAAAAAAAAqc/nEc8btwvzgI/s1600/CreateTalendJob.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-UFAvvVvNMPU/U_Bv867v3II/AAAAAAAAAqc/nEc8btwvzgI/s1600/CreateTalendJob.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i>Fig: Create a new job</i></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="text-align: left;">ii) Create a new HDFSConnection to connect the the HDFS. As this is going to connect to the sandbox which we have installed earlier, provide the IP and Port (8020) for NameNode URI.</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-b8CUyw84yXU/U_BxmsvY2II/AAAAAAAAAqo/vMEYLWJaxQY/s1600/CreateHDFSConnection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-b8CUyw84yXU/U_BxmsvY2II/AAAAAAAAAqo/vMEYLWJaxQY/s1600/CreateHDFSConnection.png" height="291" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i>Fig: Create a HDFS Connection and set the URI to the sandbox IP and Port (8020)</i></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
iii) Create a tRowGenerator object which would generate the required rows based on the schema created for it. For this example i have created 100 rows. Other information about the schema is available the below images.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-j_qT-oM8TiQ/U_B0iHZ2SqI/AAAAAAAAAq0/ZmIYIcfEJzI/s1600/tRowGenerator.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-j_qT-oM8TiQ/U_B0iHZ2SqI/AAAAAAAAAq0/ZmIYIcfEJzI/s1600/tRowGenerator.png" height="465" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<i>Fig: tRowGenerator which would generate 100 rows data.</i></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-7EfVA0XWhv0/U_B1gN09gnI/AAAAAAAAArA/tw674umc6lk/s1600/tRowGeneratorMapping.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-7EfVA0XWhv0/U_B1gN09gnI/AAAAAAAAArA/tw674umc6lk/s1600/tRowGeneratorMapping.png" height="275" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: center;">
Fig: Schema for the data generation</div>
<br />
iv) Link tHDFSConnection to tRowGenerator: Right click on the tHDFSConnection_1 and click on trigger-> On subjob ok and connect to tRowGenerator_1. This would move start the generator work after connection is successful.<br />
v) Write data to HDFS using tHDFSOutput. Right click on the tRowGenerator_1 and drag it to tHDFSOutput_1. If you need to see the generated output, you can add tLogRow_1 and connect this also in the similar way.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-xEQFHTJcNZY/U_B3uSwncNI/AAAAAAAAArM/zaMp8pgTgPw/s1600/WriteDataToHDFSUsingtHDFSOutput.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-xEQFHTJcNZY/U_B3uSwncNI/AAAAAAAAArM/zaMp8pgTgPw/s1600/WriteDataToHDFSUsingtHDFSOutput.png" height="304" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Fig: tHDFSOutput object to write to </div>
<br />
Now our design is complete, save it.<br />
<br />
d) Run/Execute job to create the file in HDFS. You can either click on the F6 or Run the job by going the Run tab and click on Run button. On Successful execution you would see the filter with the mygeneratedout.csv in the location provided in tHDFSOutput_1, which is "/user/hue/mygeneratedout.csv"<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-1Qcn04LqlPA/U_B80yqhYiI/AAAAAAAAArc/kmIpsOJDhEU/s1600/FileCreatedInHDFS.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-1Qcn04LqlPA/U_B80yqhYiI/AAAAAAAAArc/kmIpsOJDhEU/s1600/FileCreatedInHDFS.png" height="256" width="640" /></a></div>
You can click on the file to see the content.<br />
<br />
Thanks for reading the blog. :-)<br />
<br />
Soon would be write few more blogs on Hadoop and Talend which explains their specific feature in more details.<br />
<br />
<br />
<br />
<br /></div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com2tag:blogger.com,1999:blog-2415001147493991407.post-42173664436598491212014-08-10T19:47:00.000+05:302014-08-10T22:22:32.784+05:30RaspberryPi to run DC Motor using L298N Motor Controller<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.raspberrypi.org/">Raspberry Pi</a> is an awesome and interesting . I was totally astonished after knowing about it from one of my friend and colleague <b>"Alex"</b> at Misys, who keeps sharing many interesting things of this kind.<br />
<br />
I did computer application and always had this in mind that I missed to Engineering/Robotics. As I have no knowledge about electronics and mechanics so it really looked difficult for me to try something in this area. One of the main reason is the complexity involved in the Robotics in general (At least complex in my mind till i tried it).<br />
Raspberry PI has made it simple and abstract for the users like me to achieve the same.<br />
<br />
They say it is for the kids to enjoy...... I am still thinking, is it really only for kids? Any way i am enjoying this.<br />
<br />
You can find more about the <a href="http://www.raspberrypi.org/">Raspberry PI</a> from there website. What is the most amazing thing for me is that with this small credit-card sized computer it can be used to control devices. In this post i am not showing anything different than what others have done. In-fact i have also referred some of them.<br />
<br />
I have ordered the following from ebay to make motor running work for me:<br />
<br />
RaspberryPI - 1<br />
DCMotor - 1<br />
L298N Motor Controller - 1<br />
PowerSupply (2*2) - 1<br />
Male-Female and Female-Female connectors. - Few<br />
<br />
<b>RaspberryPI</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-tsnAHF68BXY/U-d7-Tnl1_I/AAAAAAAAAnY/h3KK6UkE2Kc/s1600/RaspberrPI_Image.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-tsnAHF68BXY/U-d7-Tnl1_I/AAAAAAAAAnY/h3KK6UkE2Kc/s1600/RaspberrPI_Image.jpg" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="text-align: left;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="text-align: left;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="text-align: left;"><b>DC Motor </b> </span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/--NK8_dtowLg/U-d7OEM6QSI/AAAAAAAAAnM/7rN0qPQiMrk/s1600/dcmotor.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/--NK8_dtowLg/U-d7OEM6QSI/AAAAAAAAAnM/7rN0qPQiMrk/s1600/dcmotor.JPG" height="200" width="200" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<b>L298N Motor Controller</b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-I0c-2xH9JNQ/U-d8pvvidHI/AAAAAAAAAng/9umY2o6q81w/s1600/l298n-motor-driver-board.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-I0c-2xH9JNQ/U-d8pvvidHI/AAAAAAAAAng/9umY2o6q81w/s1600/l298n-motor-driver-board.jpg" height="254" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Following video should give you some idea about RPi and what it can do.<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="315" src="//www.youtube.com/embed/zQB-AUf1dig" width="420"></iframe><br />
<br />
<br />
You can find the python program which was used to control the motor here:<br />
<br />
<script src="https://gist.github.com/deepaksinghvi/2210803e14126dcb7986.js"></script><br />
<br />
Connectivity for the pins is explained below:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-3ZFzrW0OeFc/U-d6eAyvJ2I/AAAAAAAAAm8/pumWauPEMqA/s1600/RaspberryPiL298N_H_BridgeMotorController.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-3ZFzrW0OeFc/U-d6eAyvJ2I/AAAAAAAAAm8/pumWauPEMqA/s1600/RaspberryPiL298N_H_BridgeMotorController.png" height="388" width="640" /></a></div>
<br />
<br />
Feeling excited about it by thing about the various opportunities about the application of R_PI.<br />
Already received Raspberry PI Cam, and planning to make my own security system. :-)<br />
<br />
This is the most amazing thing I have come across in the recent time.<br />
<br />
<br />
I faced specific problem with my monitor for HDMI connectivity. My first configuration was with a relatively new monitor (Dell) and it was fine. But when i switched to another monitor (LG), it did not work, there was no display. I struggled for few days but finally found the way and posted the solutions here :<br />
<br />
<a href="http://www.raspberrypi.org/forums/viewtopic.php?f=28&t=5107&p=595514">http://www.raspberrypi.org/forums/viewtopic.php?f=28&t=5107&p=595514</a><br />
<br /></div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com4tag:blogger.com,1999:blog-2415001147493991407.post-31673885706352563832013-08-28T00:29:00.000+05:302013-10-20T01:24:32.069+05:30Runtime instrumentation of bytecode using javaagent with Javassist<div dir="ltr" style="text-align: left;" trbidi="on">
I was looking into some classloading issues of websphere application server and encountered with javaagent argument , I<span style="background-color: white;"> found really interesting when i read "The agent class will be loaded by the same classloader which loads the class containing the application </span><code style="background-color: white;">main</code><span style="background-color: white;"> method.", and i started digging more into it.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">This was a new thing for me. I used the transformer to see a class loaded from which location. I was doing the same from <a href="http://deepaksinghviblog.blogspot.in/2011/06/find-from-which-jar-class-was-loaded.html">debugger for multiple class</a> was really time consuming. By using transformer i could find out easily a class was loaded by whicih classloader and from which location using ProtectionDomain, etc. Co-incidently I was working on profiling also and found it can be used there as well.</span><br />
<span style="background-color: white;"><br /></span>
I wrote small program around it and found it is useful. Many people have written already written in this area but did not find on any blog quickly that how to get a count for method execution, so i thought it writing myself. Of-course many tools like Jensor, Jrat, etc, gives lot more facility but i did it just to get method count and not any other reporting.<br />
<br />
So ultimately this blog post covers<br />
<br />
<ul style="text-align: left;">
<li>Javaagent and premain</li>
<li>ClassFileTransformer</li>
<li>Javassist</li>
</ul>
<br />
<br />
<br />
<ul>
<li><u>Javaagent and premain:</u> Introduced in JDK 5, vm argument.. Can augument java bytecode dynamically with the transformer(s) which helps in profiling, bytecode manupulation,. With this dynamic manipulation we can manipulate bytecode during the runtime and which is one of the most useful feature of java. Agent can be added in the JVM argument for your program or server. Agent must implement premain method. </li>
</ul>
A premain method is the one which gets executed before loading the class and before executing main <br />
method in the class. Javaagent can be added as the JVM argument as follows:<br />
<br />
-javaagent:<jarpath> for e.g. -javaagent:D:\Instrumentation\dsinstrumentation.jar</jarpath><br />
<br />
<br />
<script src="https://gist.github.com/deepaksinghvi/14a837fbd90e56de485d.js"></script><br />
<br />
<br />
<ul style="text-align: left;">
<li><u>ClassFileTransformer:</u> Agent can add one or more transformer, we have to implement ClassFileTransformer interface. Transformed class which is responsible for transformation of the bytecode. The transformation occurs before the class is defined by the JVM.</li>
</ul>
<br />
jar which we add as a jarpath for javaagent argument, must have Premain-Class attribute in<br />
MANIFEST.MF file. Which is as follows<br />
<br />
<script src="https://gist.github.com/deepaksinghvi/7a4e9a8115cedb5631b4.js"></script><br />
<br />
<br />
<ul style="text-align: left;">
<li><u>Javassist:</u> I have used Javassist for byte code instrumentation. We have multiple choices to achive the same. Users can use Javassist, ASM, BCEL from apache, etc.</li>
</ul>
<br />
<script src="https://gist.github.com/deepaksinghvi/6357017.js"></script><br />
<br />
You can find/download the source code from my github account repository which included dsinstrumentation.jar as well, which can be used directly.<br />
User need to provide<br />
a) -javaagent<br />
b) configuration system property which should have conf folder with the classinstrumentation.properties file<br />
with the class file name in the following format:<br />
<script src="https://gist.github.com/deepaksinghvi/6357518.js"></script><br />
<br />
<br />
<span style="color: #741b47; font-size: large;"><b>Download :</b></span> <a href="https://github.com/deepaksinghvi/instrumentation">git repository for instrumentation</a><br />
<br />
To run dsinstrumentaion.jar (which is available in the above mentioned download location) as javaagent, give the following jvm argument<br />
<br />
-javaagent:<span style="background-color: white; color: #333366; font-family: arial, helvetica, sans-serif;"><</span>FOLDER_PATH_TO_INSTRUMENTATION_JAR<span style="background-color: white; color: #333366; font-family: arial, helvetica, sans-serif;">></span>\dsinstrumentation.jar<br />
-Dconfiguration=<span style="background-color: white; color: #333366; font-family: arial, helvetica, sans-serif;"><</span>FOLDER_PATH_TO_CONF_FOLDER<span style="background-color: white; color: #333366; font-family: arial, helvetica, sans-serif;">>\</span>conf</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-29859414865307772072013-01-25T20:16:00.001+05:302013-10-20T01:27:02.401+05:30Realization of dream of multitenancy with Google App Engine and MongoDB on cloud.<div dir="ltr" style="text-align: left;" trbidi="on">
<b>Multitenancy </b>and the problem of managing the data<br />
<br />
Multitenancy refers to a principle in software architecture where a single instance of the software runs on a server, serving multiple client organizations (tenants). Multitenancy is contrasted with a multi-instance architecture where separate software instances (or hardware systems) are set up for different client organizations. With a multitenant architecture, a software application is designed to virtually partition its data and configuration, and each client organization works with a customized virtual application instance. [<b><a href="http://en.wikipedia.org/wiki/Multitenancy">WIKI</a></b>]<br />
<br />
What if i want to migrate my existing system to any cloud engine?<br />
Move all of code/business logic to app engine, perfect, no problem, but what if my database is already on another physical infrastructure(and which is shredded,etc)?<br />
<br />
<br />
Of-course app engine/GAE gives you the ability to scale, security, high availability, etc. but with their own terms.You need to use the database which is supported only by the app engine.<br />
Which is one of the major drawback for the GAE and similarly other cloud engines.<br />
<br />
I wanted to use MongoDB with Google App Engine(GAE) and could not see the possibility because we can't run MongoDB on GAE and supposed to use Google datastore.<br />
<br />
There are few more issues as GAE's highly restricted sandbox. As GAE's docs says your application can only access other computers on the internet through the provided URL fetch and email services (and fewer more ways). Other computers can only connect to the application by making HTTP/HTTPS requests on the standard port. If you want to open a socket from your business logic, its not allowed and GAE will raise an exception.<br />
<br />
Few good things happened in this direction, when google annoucned back in 2011 about Google Cloud SQL webservice to support MySQL - <a href="https://developers.google.com/cloud-sql/">https://developers.google.com/cloud-sql/</a><br />
Some of the other cloud engines have provided more options such as Amazon RDS supports MySQL, Oracle or Microsoft SQL Server database engine. This means that the code, applications, and tools you already use today with your existing databases can be used with Amazon RDS.<br />
<br />
I was adamant to move from MongoDB and to use my application with the GAE and <a href="https://mongolab.com/">mongolab </a>became the savior. Why i want to use my own choice of database because i have already have my own hosted database servers because i want to manage data on my own (security reason, business reason, etc)<br />
<br />
Here is the steps how can we use GAE with MongoDB, when your MongoDB instance is hosted on DB cloud.<br />
<br />
1) Signup, login and create a database on <a href="https://mongolab.com/">mongolab</a> <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-iLzaO78jGMo/UQFnBq88LbI/AAAAAAAAAgI/XoN9CH7Erwg/s1600/CreateDatabase.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="http://2.bp.blogspot.com/-iLzaO78jGMo/UQFnBq88LbI/AAAAAAAAAgI/XoN9CH7Erwg/s400/CreateDatabase.JPG" width="281" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
This is going to create mongodb as the database.</div>
<br />
<br />
2) Create a collection<br />
Creation of collection (table) can be done by going to database and click on "Add".<br />
<br />
3) Open collection which is currently empty and open the API view. <br />
Connection information on the top of the "Collections" tab is the url which we are going to use interact<br />
with "mongodb" database created in step1, mongolab has provided REST based methods for database<br />
interactions.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-r0Zn_X90p0o/UQFnBLgFZcI/AAAAAAAAAgE/XioqAhDufMQ/s1600/ApiView.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="132" src="http://2.bp.blogspot.com/-r0Zn_X90p0o/UQFnBLgFZcI/AAAAAAAAAgE/XioqAhDufMQ/s640/ApiView.JPG" width="640" /></a></div>
<br />
At this point we are done with our database part.<br />
<br />
Lets move to GWT(Google Web Toolkit) to create a project.<br />
<div>
<br /></div>
<div>
4) Create a Web Application Project and make sure that "<b>Use Google App Engine</b>" is checked.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-LcoTxu_zDBE/UQFm_CueyGI/AAAAAAAAAf8/fhD5x06qlss/s1600/CreateProjectGae.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="http://2.bp.blogspot.com/-LcoTxu_zDBE/UQFm_CueyGI/AAAAAAAAAf8/fhD5x06qlss/s640/CreateProjectGae.JPG" width="473" /></a></div>
<br />
Create a GWT based application, i have used the almost same application which i have created earlier,<br />
you can refer the steps for the application creation here <a href="http://deepaksinghviblog.blogspot.in/2012/07/gwt-rpc-server-communication-with.html" style="color: #6699cc; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; text-decoration: initial;"><b>GWT RPC - Server communication with </b></a><br />
<b style="color: #6699cc; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; text-decoration: initial;"> <a href="http://deepaksinghviblog.blogspot.in/2012/07/gwt-rpc-server-communication-with.html" style="color: #6699cc; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; text-decoration: initial;">MongoDB</a></b><br />
<br />
The only change here is that the mongodb is hosted on some cloud, and because of that implementation<br />
for is going to change.<br />
<br />
5) Insert a recod and retrive records from cloud database.<br />
Inserting a record into the databased using the apache's HTTPClient to connect to mongodb hosted on<br />
cloud, which interacts with the REST based webservice.<br />
<script src="https://gist.github.com/4631878.js"></script><br />
<br />
<br />
Get all available users from the database.<br />
<script src="https://gist.github.com/4631919.js"></script><br />
<br />
As i mentioned earlier about the sandbox security of GAE, i could not use HttpClient directly to invoke<br />
webserivce and get the data, and had to tweak it bit to make it happen. Shortly going to update about the<br />
problem as well in this post, which will be describing about GAEConnectionManager.<br />
<br />
6) In order to deploy the developed application on cloud, create an an application identifier on app engine. <br />
After creation you can see the application as follows:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-15r1RRWlmk4/UQKPuAGTagI/AAAAAAAAAgc/JJ0Nlw8W9Cs/s1600/Creating_app_GAE.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="116" src="http://4.bp.blogspot.com/-15r1RRWlmk4/UQKPuAGTagI/AAAAAAAAAgc/JJ0Nlw8W9Cs/s640/Creating_app_GAE.JPG" width="640" /></a></div>
<br />
<br />
Update in <webapplicationproject>/war/WEB-INF/appengine-web.xml and place it under </webapplicationproject>"application" tag.<br />
<br />
<script src="https://gist.github.com/4634673.js"></script><br />
<br />
7) Login to google account from eclipse(you can find it in the bottom left) and deploy the app on the app<br />
engine.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-bHW2yyhFU9A/UQKRe2bTVgI/AAAAAAAAAgs/G4zeJzgHsxc/s1600/DeployApp.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-bHW2yyhFU9A/UQKRe2bTVgI/AAAAAAAAAgs/G4zeJzgHsxc/s1600/DeployApp.JPG" /></a></div>
<br />
<br />
And we are <span style="color: #38761d;"><b>done</b></span><span style="background-color: white;"><b><span style="color: #38761d;">!</span>,<span style="color: #38761d;"> </span></b></span> app is available and hosted.<br />
You can access it from here: <a href="http://mongocloudapp.appspot.com/">http://mongocloudapp.appspot.com/</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-6HkJ5fwsVFs/UQKatzYvHWI/AAAAAAAAAg8/ZzJViAMULOQ/s1600/deployedApp.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="311" src="http://4.bp.blogspot.com/-6HkJ5fwsVFs/UQKatzYvHWI/AAAAAAAAAg8/ZzJViAMULOQ/s400/deployedApp.JPG" width="400" /></a></div>
I would like to say that overall it is a very nice experience with the app development with GAE and<br />
without loosing the my own preferred database.</div>
</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-85338582099425327302012-11-17T23:26:00.001+05:302012-11-17T23:39:26.495+05:30SQLite database on Android platform<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
Android provides nice way of storing data into database and this is possible with its internal library SQLite. SQLite is a very light weight database and its included into android's library stack.<br />
See android architecture <a href="http://www.slideshare.net/Gemsbond/fix-protocol-trading-and-mobile-compu/13">here</a> for more details.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-VCr4Q3tyyJU/UKfSGUC7yWI/AAAAAAAAAfE/OHUzuERnc1o/s1600/sqlite370_banner.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="91" src="http://3.bp.blogspot.com/-VCr4Q3tyyJU/UKfSGUC7yWI/AAAAAAAAAfE/OHUzuERnc1o/s200/sqlite370_banner.gif" width="200" /></a> <span style="font-size: x-large;"><b>+</b></span> <a href="http://1.bp.blogspot.com/-_p5f5OUD5M4/UKfSe0PoxkI/AAAAAAAAAfM/eZyiOpETF_0/s1600/android-jellybean.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="161" src="http://1.bp.blogspot.com/-_p5f5OUD5M4/UKfSe0PoxkI/AAAAAAAAAfM/eZyiOpETF_0/s200/android-jellybean.png" width="200" /></a></div>
<br />
SQLite is opensource database, to learn more about the SQLite refer <a href="http://www.sqlite.org/sqlite.html">here</a>.<br />
<br />
This example demonstrate how SQLite can be used in Android application.<br />
I have used few basic example of create database and table, insert record, select record from table and delete table.<br />
<br />
This blog also explains the example for the android basic widgets which includes TextView and buttons. If user wants they can use other external tool for the development of gui ( for eg. droiddraw , etc), i have have done most of the gui development by modifying the xml directly or with the default gui builder.<br />
<br />
<b>Create Database</b><br />
<script src="https://gist.github.com/4097846.js"> </script><br />
<br />
<b>Create Table</b><br />
<script src="https://gist.github.com/4097867.js"> </script><br />
<br />
<b>Insert record into table</b><br />
<script src="https://gist.github.com/4097893.js"> </script><br />
<br />
<b>Query SQLite table</b><br />
<script src="https://gist.github.com/4097903.js"> </script><br />
<br />
<b>Delete table and close database connection</b><br />
<script src="https://gist.github.com/4097909.js"> </script><br />
<br />
<br />
<br />
This is a simple android which which shows a thought of the day to a user, if user wants to see more thoughts than user can press next else user can press thanks button. Purpose of this post is just to show the capability of SQLite, so there are only 20 records available in the table and they are shown in some random number.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-6mBthBmFdM4/UKfLZhNDHfI/AAAAAAAAAe0/XmhfEpJfGAU/s1600/ThoughtOfTheDayAndroidApp.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="http://3.bp.blogspot.com/-6mBthBmFdM4/UKfLZhNDHfI/AAAAAAAAAe0/XmhfEpJfGAU/s400/ThoughtOfTheDayAndroidApp.png" width="271" /></a></div>
<br />
<br />
<br />
You can either checkout or download this sample application code from <a href="https://github.com/deepaksinghvi/SQLiteWithAndorid">here</a>.</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-13041045834567881692012-08-06T00:58:00.000+05:302013-10-20T01:27:29.003+05:30GWT Editable Table - CellTable with remove row<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
GWT offers table where we can specify column types, and make rows data modifiable.<br />
<br />
CellTable supports paging and columns, i have not used paging in this example but only specified the columns for text and button.
<br />
<br />
<br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<script src="https://gist.github.com/3266674.js"> </script></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
</div>
The <code>Column</code> class defines the <code>Cell</code> used to render a column.<br />
<br />
Implement <code>Column.getValue(Object)</code> to retrieve the field value from the row object that will be rendered in the <code>Cell</code>. <br />
<br class="Apple-interchange-newline" />
<br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
and the data which will be rendered:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-FtIb7YasW5U/UB7B1LAia8I/AAAAAAAAAd4/zCAO6nB0Hh8/s1600/celltable.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="241" src="http://2.bp.blogspot.com/-FtIb7YasW5U/UB7B1LAia8I/AAAAAAAAAd4/zCAO6nB0Hh8/s400/celltable.png" width="400" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
You can use TextCell instead of EditTextCell, in case you do not want to make column editable.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-UYIkKP38wGg/UB7FuD-Cr1I/AAAAAAAAAeI/czlPNJp6r7s/s1600/EditCellTable.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="252" src="http://3.bp.blogspot.com/-UYIkKP38wGg/UB7FuD-Cr1I/AAAAAAAAAeI/czlPNJp6r7s/s400/EditCellTable.png" width="400" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
DataProvider, ListDataProvider can display data which should be provided as list.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I have used modal User to create few items in the list and provide the same to the ListDataProvider and which works as the modal for the celltable.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<script src="https://gist.github.com/3266719.js"> </script></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Remove row from the dataProvider and table.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<script src="https://gist.github.com/3266734.js"> </script></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
When user clicks on "x" update gets called from updater and where we can remove the row from the modal and refresh modal and re-draw table (i found without refresh modal and redraw table call also it was working fine).</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-cdxVlMf0iBg/UB7F0ApnXII/AAAAAAAAAeQ/pOfLHXt1hiM/s1600/DeleteRowCellTable.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="222" src="http://3.bp.blogspot.com/-cdxVlMf0iBg/UB7F0ApnXII/AAAAAAAAAeQ/pOfLHXt1hiM/s400/DeleteRowCellTable.png" width="400" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Example files are available to <span style="color: purple;"><b><a href="https://sites.google.com/site/deepaksinghvi/Home/gwt">download</a> <<==== click here to download</b></span>.</div>
<div style="text-align: left;">
Look for <span style="background-color: #f5f3f2; color: #414b56; font-family: Arial, Verdana, sans-serif; font-size: 12px;">GwtExamples.rar (v.1)</span><span style="background-color: #f5f3f2; color: #414b56; font-family: Arial, Verdana, sans-serif; font-size: 12px;"> </span></div>
</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com6tag:blogger.com,1999:blog-2415001147493991407.post-64622998062345916742012-07-24T10:15:00.001+05:302013-10-20T01:27:52.108+05:30GWT RPC - Server communication with MongoDB<div dir="ltr" style="text-align: left;" trbidi="on">
Further to the client <a href="http://deepaksinghviblog.blogspot.in/2012/07/google-web-toolkit-step-by-step-gwt.html">tutorial </a>in the previous blog, here i am going to present how client communicates with the server with the help of GWT RPC (Remote Procedure Call).<br />
Using RPC which works asynchronously only specific part of the client components can be updated without updating the complete client.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-R_GCx838-bM/UA7caHRJ64I/AAAAAAAAAds/jRLnAxHIuVo/s1600/images.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="http://3.bp.blogspot.com/-7ev6ZZxsJH8/UA7cXOIFjbI/AAAAAAAAAdk/CR0cCtGWc_0/s1600/logo-gwt.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="182" src="http://3.bp.blogspot.com/-7ev6ZZxsJH8/UA7cXOIFjbI/AAAAAAAAAdk/CR0cCtGWc_0/s200/logo-gwt.png" width="200" /></a><img border="0" height="66" src="http://2.bp.blogspot.com/-R_GCx838-bM/UA7caHRJ64I/AAAAAAAAAds/jRLnAxHIuVo/s200/images.jpg" width="200" /></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Pre-Requisite:<br />
Java, Eclipse with <span style="color: #3d85c6;">GWT </span>plugin, <span style="color: purple;">MongoDB</span>.<br />
<br />
<br />
<br />
In this example i have extended the previous client to<br />
a) save data i.e. user data to database and<br />
b) authentication user<br />
<br />
Client can send/receive serializable objects to/from the server. In this example server is going to use that data and insert in into DB, which is MongoDB in our case.<br />
<br />
<br />
<u><b><span style="color: #0b5394;">Create Modal</span></b></u><br />
As the client can send serializable java objects over HTTP. Lets create a modal first.<br />
<br />
<br />
User.java<br />
<script src="https://gist.github.com/3167902.js">
</script><br />
<br />
<br />
<u><b><span style="color: #0b5394;">Create Service</span></b></u><br />
Now we need to have a service which a client can use.<br />
In our case we name it as MongoDBService and it implements RemoteService<br />
<br />
<br />
MongoDBService.java<br />
<script src="https://gist.github.com/3167852.js">
</script><br />
<br />
<br />
@RemoteServiceRelativePath("dbservice") is annotation used for the service identification and calling, which should match with the service path defined in web.xml for servlet-mapping tag.<br />
<br />
<span style="background-color: white;">We need to create Asynchronous service for our MongoDBService, client is actually going to call RPC through MongoDBServiceAsync. All methods of </span><span style="background-color: white;">MongoDBService will have extra parameter which is of type </span><span style="background-color: white;">AsyncCallback. Client call will be notified when any asynchronous service call completes.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">MongoDBServiceAsync.java</span><br />
<span style="background-color: white;"><script src="https://gist.github.com/3167858.js">
</script></span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><u><br /></u></span>
<span style="background-color: white; color: #0b5394;"><u><b>Implement service at the server side</b></u></span><br />
<span style="background-color: white;">Create class MongoDBServiceImpl which extends RemoteServiceServlet and implements MongoDBService. Basically </span><span style="background-color: white;">MongoDBServiceImpl</span><span style="background-color: white;"> is a servlet which is extending from RemoteServiceServlet rather than the HttpServlet directly.</span><br />
<span style="background-color: white;"><br /></span>
<br />
<span style="background-color: white;"><u><br /></u></span>
<span style="background-color: white; color: #0b5394;"><u><b>Updating web.xml with servlet details</b></u></span><br />
<span style="background-color: white;">make sure that all the service which are created with the annotation RemoteServiceRelativePath are added properly for each servlet.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">web.xml</span><br />
<span style="background-color: white;"><script src="https://gist.github.com/3167873.js">
</script></span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><br /></span>
url pattern tag path should be formed using module/service.<br />
In this case module name is gwtwebappdemo ( see in GwtWebAppDemo.gwt.xml for rename-to value) and service in dbservice.<br />
<br />
<br />
<b><u><span style="color: #0b5394;">Client Service Call</span></u></b><br />
<span style="background-color: white;">calling service from client
</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><script src="https://gist.github.com/3167849.js">
</script></span><br />
<br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><b><u><span style="color: #0b5394;">Running application</span></u></b>
</span><br />
<span style="background-color: white;"><b><u><br /></u></b></span>
<span style="background-color: white;">1) Start MongoDB</span><br />
<span style="background-color: white;"> In this case i have started DB without authentication and on default port.</span><br />
<span style="background-color: white;"> Default port is 27017 which we have used to connect to database in the code. See DBUtil.java in the</span><br />
<span style="background-color: white;"> example.</span><br />
<span style="background-color: white;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-1CSKkA6PykU/UA4dzFc_dOI/AAAAAAAAAdA/fjbvsw6tOwE/s1600/mongod_start_defaultport.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="155" src="http://4.bp.blogspot.com/-1CSKkA6PykU/UA4dzFc_dOI/AAAAAAAAAdA/fjbvsw6tOwE/s640/mongod_start_defaultport.JPG" width="640" /></a></div>
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">2) Add user into database.</span><br />
<span style="background-color: white;"> Client calls asynchronous call to server, server starts the processing. Once request processing is finished it calls back with the call back handler provided in the request. Client gets call in onFailure in case of failure and onSuccess in case of request processing successful.</span><br />
<span style="background-color: white;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-Lad81ss4ffY/UA4dmKB52zI/AAAAAAAAAc4/84D0_JWgh18/s1600/addUser.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="340" src="http://3.bp.blogspot.com/-Lad81ss4ffY/UA4dmKB52zI/AAAAAAAAAc4/84D0_JWgh18/s400/addUser.JPG" width="400" /></a></div>
<span style="background-color: white;"><br /></span>
In the server side, server received the User object and sets the data into User collection which is in the mymongodb database using BasicDBObject.<br />
<br />
MongoDBServiceImpl.java<br />
<script src="https://gist.github.com/3167868.js">
</script><br />
<br />
3) Check the collection ( table) creation in dbs (database)<br />
In this example mymongodb is the database and User is the collection which we are using.<br />
Initially both database and collection is not present, when the first save happens. Both gets automatically <br />
created.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-ANyKeCY8xEY/UA4eBmY8zwI/AAAAAAAAAdI/UPMYp-uPONA/s1600/verifyUserAdditionInDB.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="216" src="http://2.bp.blogspot.com/-ANyKeCY8xEY/UA4eBmY8zwI/AAAAAAAAAdI/UPMYp-uPONA/s640/verifyUserAdditionInDB.JPG" width="640" /></a></div>
<br />
<span style="background-color: white;"><br /></span>
4) Check User Authentication with wrong input.<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="http://2.bp.blogspot.com/-dkqPAA7yS2w/UA4eeiksQmI/AAAAAAAAAdY/dKKtSLvutfs/s1600/userauthenticationFailed.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="363" src="http://2.bp.blogspot.com/-dkqPAA7yS2w/UA4eeiksQmI/AAAAAAAAAdY/dKKtSLvutfs/s400/userauthenticationFailed.JPG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #0b5394;">Download complete example</span> from <a href="https://sites.google.com/site/deepaksinghvi/Home/gwt">here</a> in <a href="https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxkZWVwYWtzaW5naHZpfGd4OjEwYTU2NDBhYzUyZGY5YjY" style="background-color: #f5f3f2; border-bottom-style: none !important; color: #551a8b; font-family: Arial, Verdana, sans-serif; font-size: 12px; line-height: 40px; text-align: -webkit-auto; text-decoration: none !important; white-space: nowrap;">GwtWebAppDemo_server.rar</a><span style="background-color: #f5f3f2; font-family: Arial, Verdana, sans-serif; font-size: 12px; line-height: 40px; text-align: -webkit-auto; white-space: nowrap;"> file.</span></div>
<br /></div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com1tag:blogger.com,1999:blog-2415001147493991407.post-86566759605768249702012-07-17T20:53:00.000+05:302012-07-19T09:52:28.973+05:30Google Web Toolkit - step by step GWT client tutorial<div dir="ltr" style="text-align: left;" trbidi="on">
Its free!<br />
Its opensource!<br />
Its makes development faster!<br />
Its amazing!<br />
Faster development of web application with an ease.<br />
AJAX based app develop in java and run in all browser.<br />
Works well with Android/iPhone.<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="http://3.bp.blogspot.com/-aymC13euXoc/UAeLa5Rz9FI/AAAAAAAAAco/Dj59nHQY430/s1600/images.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-aymC13euXoc/UAeLa5Rz9FI/AAAAAAAAAco/Dj59nHQY430/s1600/images.jpg" /></a></div>
<br />
Google has done preaching on their site and blog about GWT, find more details their.<br />
Ofcourse believers of those preaching also wrote a lot about it on their blog(s), website(s), papers, etc,<br />
So, m<span style="background-color: white;">oving to example.</span><span style="background-color: white;">:</span><br />
<br />
Pre-requisetes<br />
Eclipse, Java.<br />
<br />
<br />
Download and install GWT plugin for eclipse<span style="background-color: white;">, in my case its for indigo and link is as follows:</span><br />
http://dl.google.com/eclipse/plugin/3.7<br />
<br />
for complete instruction use the following link<br />
<a href="https://developers.google.com/eclipse/docs/install-eclipse-3.7">https://developers.google.com/eclipse/docs/install-eclipse-3.7</a>
<br />
<br />
<br />
Create a Google Web Project -> GwtWebAppDemo<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-CAR5qtod9Nk/UAV8tPbyjdI/AAAAAAAAAbk/VBvnZ7tTubg/s1600/SelectGoogleWebProject.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="http://4.bp.blogspot.com/-CAR5qtod9Nk/UAV8tPbyjdI/AAAAAAAAAbk/VBvnZ7tTubg/s640/SelectGoogleWebProject.JPG" width="640" /></a></div>
<br />
Click on next.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-iJTbm78AlCk/UAV_SKMhwII/AAAAAAAAAb0/OkIU_7nLxbE/s1600/CreateGoogleWebProject.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="http://4.bp.blogspot.com/-iJTbm78AlCk/UAV_SKMhwII/AAAAAAAAAb0/OkIU_7nLxbE/s640/CreateGoogleWebProject.JPG" width="472" /></a></div>
<br />
Un-check the checkbox "Use Google App Engine" as we are going to use internal app server.<br />
Click on finish.<br />
A new google web project is created with default project files.<br />
<br />
Delete all default java files from client, server and shared package.<br />
<span style="background-color: white;"> Add a new Entry-Point class into the client package.</span><br />
<span style="background-color: white;"><br /></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-aUczppOmAiM/UAV9bH3b5cI/AAAAAAAAAbs/vCfuF4WfY0w/s1600/EntryPointClass.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="http://1.bp.blogspot.com/-aUczppOmAiM/UAV9bH3b5cI/AAAAAAAAAbs/vCfuF4WfY0w/s640/EntryPointClass.JPG" width="640" /></a></div>
<br />
One entry will be added into GwtWebAppDemo.gwt.xml.<br />
Verify that entry point is correct in GwtWebAppDemo.gwt.xml. In this case its<br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;"><script src="https://gist.github.com/3130287.js?file=GwtWebAppDemo.gwt.xml">
</script></span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">Change the class in entry-point tag if you have changed/added the class manually and not as Entry Point Class.</span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">Add RootPanel to the entry point class i.e. </span><span style="background-color: white;">GwtDemoAppClient.java</span><span style="background-color: white;"> </span><span style="background-color: white;">in onModuleLoad() method</span><span style="background-color: white;">.</span><br />
<span style="background-color: white;"></span><br />
<br />
<br />
<pre><b>RootPanel rootPanel = RootPanel.get();</b>
</pre>
<br />
RootPanle is a panel to which all other widgets must ultimately be added.<br />
RootPanels are never created directly. Rather, they are accessed via get().<br />
Most applications will add widgets to the default root panel in their EntryPoint.onModuleLoad() methods.<br />
<br />
<span style="background-color: white;"></span><br />
<br />
After that right client on GwtDemoAppClient.java and open with GWT Designer<br />
We are ready for adding the components by drag and drop.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-tNNs6HZTQ8k/UAV_mCr-iPI/AAAAAAAAAb8/JSZpPuSQ6OQ/s1600/AddLayoutPanelToRootPanel.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="553" src="http://4.bp.blogspot.com/-tNNs6HZTQ8k/UAV_mCr-iPI/AAAAAAAAAb8/JSZpPuSQ6OQ/s640/AddLayoutPanelToRootPanel.JPG" width="640" /></a></div>
<br />
I have selected LayoutPanel as it allows its children to be positioned using arbitrary constraints.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-tyRooL-ZR94/UAWB81pzrVI/AAAAAAAAAcc/64AW916MdMM/s1600/AddComponents.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="553" src="http://1.bp.blogspot.com/-tyRooL-ZR94/UAWB81pzrVI/AAAAAAAAAcc/64AW916MdMM/s640/AddComponents.JPG" width="640" /></a></div>
<div>
<br /></div>
<div>
After adding the components you can see the generated code:</div>
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;"><script src="https://gist.github.com/3130257.js">
</script></span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">Add few componets to the layoutPanel and test the demo without web app to see it as preview.</span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-gmJpHDOmCl4/UAWAWhdNRkI/AAAAAAAAAcE/NgFxJNsoQKo/s1600/RunDemo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="553" src="http://2.bp.blogspot.com/-gmJpHDOmCl4/UAWAWhdNRkI/AAAAAAAAAcE/NgFxJNsoQKo/s640/RunDemo.JPG" width="640" /></a></div>
<br />
<br />
If its fine than go ahead and create a web app/deploy to internal app server which is jetty.<br />
Else go and add/remove/update components in the panel.<br />
<br />
Change the default html content of GwtWebAppDemo.html in the war/WEB-INF folder.<br />
<br />
<br />
<br />
Right click on the project and click on Run As Web Application<br />
It will start new tab as development mode, copy the link and paste into your browser<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-Fxf8P5-6vLU/UAWAlhLC7bI/AAAAAAAAAcM/J5zMsg9eHEE/s1600/SelectTheLink.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="148" src="http://4.bp.blogspot.com/-Fxf8P5-6vLU/UAWAlhLC7bI/AAAAAAAAAcM/J5zMsg9eHEE/s640/SelectTheLink.JPG" width="640" /></a></div>
<br />
<br />
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div>
For the first time it will ask for installing the plugin for gwt. </div>
<div>
Install it and see the application running.</div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-vWFbd0GGAlg/UAWBOiwvr0I/AAAAAAAAAcU/3vH16xfq-Ic/s1600/RunningWebApp.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="409" src="http://4.bp.blogspot.com/-vWFbd0GGAlg/UAWBOiwvr0I/AAAAAAAAAcU/3vH16xfq-Ic/s640/RunningWebApp.JPG" width="640" /></a></div>
<div>
<br /></div>
<div>
Do remember to close the client running the the IDE before launching it again.
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
You can download sample from here --> <a href="https://sites.google.com/site/deepaksinghvi/Home/gwt"><span style="color: #551a8b; font-family: Arial, Verdana, sans-serif;"><span style="background-color: #f5f3f2; font-size: 12px; line-height: 40px; text-align: -webkit-auto; white-space: nowrap;">GwtWebAppDemo.rar</span></span><span style="background-color: #f5f3f2; font-family: Arial, Verdana, sans-serif; font-size: 12px; line-height: 40px; text-align: -webkit-auto; white-space: nowrap;"> </span></a></div>
</div>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com5tag:blogger.com,1999:blog-2415001147493991407.post-45190386179447094652012-03-28T23:55:00.000+05:302014-01-15T20:33:25.146+05:30Power of Script writing in the world of JavaScript.<div dir="ltr" style="text-align: left;" trbidi="on">
When i was intern, i got a change to choose the technology on which i could develop a tool.
<span style="color: #0b5394;">PHP </span>or Java. I choose Java, why.... hmmm... its easy to learn, use, understand, scalable (a myth), etc, etc, etc.<br />
Later understood facebook uses php a lot.<br />
<br />
Browsed few <span style="color: #274e13;">perl </span>scripts few years back and did not even like it.<br />
Later understood perl developed is easiest in many ways and faster. Faster in execution too.
<br />
<br />
JavaScript........ahhhh, what is this, i did not like it. Some kind of client side work, which looks like always messy. :-(<br />
Later understood that its not only the client side script any more.<br />
<br />
So suddenly my thinking about the script writing, understanding, acceptance has changed.
Script programming is amazing.<br />
<br />
First got to know about <span style="color: purple;"><b>MONGODB</b></span>. Oh its amazing, it understands java script. We can do all crud<br />
operations with java script syntax. Its a document oriented storage with "<span style="color: #cc0000;"><b>NOSQL</b></span>".<br />
<br />
Than checked which mapper can be used and found node.js -> mongoose.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-g2WuLZs7IVQ/T3b67lUt1WI/AAAAAAAAAZg/TAhQMijYu-0/s1600/nodejs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="64" src="http://3.bp.blogspot.com/-g2WuLZs7IVQ/T3b67lUt1WI/AAAAAAAAAZg/TAhQMijYu-0/s200/nodejs.png" width="200" /></a></div>
<br />
And tried some hands on with <a href="http://nodejs.org/">node.js</a><span id="goog_313887877"></span><span id="goog_313887878"></span><a href="http://www.blogger.com/"></a> + <a href="http://www.mongodb.org/">mongodb</a> + <a href="http://expressjs.com/">express</a> + <a href="http://mongoosejs.com/">mongoose</a> + <a href="http://jade-lang.com/">jade</a> (a template engine).
Jade for view, remaining things for model and controller.<br />
<br />
And commonality in all these things are java script is everywhere.
Commonality, an important factor for a software product with quality. Of-course there are many point and which i do not want to touch now.<br />
<div style="text-align: center;">
So the point is: </div>
<div style="text-align: center;">
<span style="color: #b45f06;">Server side java script. </span></div>
<div style="text-align: center;">
<span style="color: #b45f06;">Client side java script. </span></div>
<div style="text-align: center;">
<span style="color: #b45f06;">Database operations in java script. </span></div>
<div style="text-align: center;">
<span style="color: #134f5c;">Oh, its heaven --> IT IS THE WORLD OF JAVA SCRIPT.</span></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
I am going to take you through some steps which can be useful to start with server side java script. You can download the project from my git repository at <br />
<a class="site-logo" href="https://github.com/" style="background-color: white; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #4183c4; float: left; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 9px; position: relative; text-align: -webkit-auto; text-decoration: none; zoom: 1;"><img alt="GitHub" class="github-logo-4x" height="30" src="https://a248.e.akamai.net/assets.github.com/images/modules/header/logov7@4x.png?1323882770" style="border-bottom-width: 0px; border-color: initial; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-style: initial; border-top-width: 0px; float: left; font-size: 13px; font: inherit; left: 0px; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: absolute; top: 10px;" /><img alt="GitHub" class="github-logo-4x-hover" height="30" src="https://a248.e.akamai.net/assets.github.com/images/modules/header/logov7@4x-hover.png?1324325405" style="border-bottom-width: 0px; border-color: initial; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-style: initial; border-top-width: 0px; font-size: 13px; font: inherit; left: 0px; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; opacity: 0; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: absolute; top: 10px;" /></a><br />
<div class="topsearch " style="background-color: white; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-color: rgb(250, 250, 250); border-left-style: solid; border-left-width: 1px; border-right-width: 0px; border-style: initial; border-top-width: 0px; box-shadow: rgb(224, 224, 224) -1px 0px 0px; clear: none; color: #333333; float: left; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 0px; margin-left: 87px; margin-right: 0px; margin-top: 0px; padding-bottom: 10px; padding-left: 10px; padding-right: 10px; padding-top: 10px; text-align: -webkit-auto; width: auto;">
<form accept-charset="UTF-8" action="https://github.com/search" id="top_search_form" method="get" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; float: left; font-size: 13px; font: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
<a class="advanced-search tooltipped downwards" href="https://github.com/search" style="-webkit-transition-delay: 0ms; -webkit-transition-duration: 0.15s; -webkit-transition-property: opacity; -webkit-transition-timing-function: ease-in; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #4183c4; float: right; font-size: 13px; font: inherit; height: 16px; margin-bottom: 0px; margin-left: 5px; margin-right: 0px; margin-top: 4px; opacity: 0.3; overflow-x: hidden; overflow-y: hidden; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-decoration: none; width: 16px;" title="Advanced Search"><span class="mini-icon advanced-search" style="background-color: transparent; background-image: url(https://a248.e.akamai.net/assets.github.com/images/modules/header/userbox/icons.png); background-position: -120px 0px; background-repeat: no-repeat no-repeat; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; display: inline-block; font: inherit; height: 16px; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: text-bottom; width: 16px;"></span></a></form>
</div>
<br />
<a href="https://github.com/deepaksinghvi/nodejssample">https://github.com/deepaksinghvi/nodejssample</a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
1) Download Install node from <a href="http://nodejs.org/">http://nodejs.org/</a>. This installs node and npm ( node package manager).</div>
<div style="text-align: left;">
npm is used to install all the dependent modules, like mongoose, express, jade, stylus, etc.<br />
2) Next install all the dependent modules:</div>
<div style="text-align: left;">
<span style="color: #e06666;"><b>cmd prompt> npm install mongoose express jade stylus</b></span></div>
<div style="text-align: left;">
or you can do one by one also.</div>
<div style="text-align: left;">
3) Now lets write few java script files</div>
<div style="text-align: left;">
a) load all the required libraries which we have just installed and create an application server.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<script src="https://gist.github.com/2228503.js">
</script></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
b) create model and controller scripts.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Creating a model (models/model.js).</div>
<div style="text-align: left;">
<script src="https://gist.github.com/2228405.js">
</script></div>
<div style="text-align: left;">
The controller part normally known in the node.js development worlds as routing, i have kept in the main script file only ( which is app.js).</div>
<div style="text-align: left;">
Routing of creation of a new user. This creates a new user and saved it into the database.</div>
<div style="text-align: left;">
<script src="https://gist.github.com/2228479.js">
</script></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
User clicks on "Create User" button and it request gets routed to above mentioned script.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-V522xol_tJw/T3NT3U9XHQI/AAAAAAAAAZM/X9WEJI714Eo/s1600/createNewUser.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-V522xol_tJw/T3NT3U9XHQI/AAAAAAAAAZM/X9WEJI714Eo/s1600/createNewUser.png" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Once user gets added than listing of users shown from the table ( known as collection in the world of mongodb) or it can be shown if user clicks on "Show Users" link in the above image.</div>
<div style="text-align: left;">
<script src="https://gist.github.com/2228645.js">
</script></div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-A58LADVbXaU/T3NUAT8HqII/AAAAAAAAAZU/m_14xH9JlAM/s1600/userList.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="http://2.bp.blogspot.com/-A58LADVbXaU/T3NUAT8HqII/AAAAAAAAAZU/m_14xH9JlAM/s320/userList.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
4) Enjoy the sample application.</div>
<div class="separator" style="clear: both; text-align: left;">
To start the application:</div>
<div class="separator" style="clear: both; text-align: left;">
a) Start database</div>
<div class="separator" style="clear: both; text-align: left;">
<b> <span style="color: #e06666;">mongodb/bin>mongod --dbpath <yourpath>/data/db</yourpath></span></b><span style="color: #e06666;"> </span></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div class="separator" style="clear: both; text-align: left;">
b) Start sample application</div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #e06666;"><b>nodejssample> node app.js</b></span></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I tried to provide a very simple sample to start with node.js and mongodb. Its a perfect match for a rapid web based development. </div>
<div class="separator" style="clear: both; text-align: left;">
WebSocket also looks very promising, a lot has been written about it. I would like to try some of the exercise on that and will be sharing that as well in the next blog (hopefully).</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com1tag:blogger.com,1999:blog-2415001147493991407.post-3480011063442806762011-12-03T09:29:00.001+05:302011-12-04T17:06:10.619+05:30QuickFix/J based Fix Protocol enabled trading client<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
Purpose of this post is to make available a simple sample application for the people who either wants to start learning or want to develop of FIX (FIX protocol) based trading client. <br />
<br />
FIX Acceptor ( which acts as the Stock Exchange)<br />
FIX Initiator ( which acts as the Trading Client)<br />
<br />
Its just a simple client without any GUI.<br />
People who are already aware about the FIX and just want to try the sample app in java can jump directly to the <a href="https://sites.google.com/site/deepaksinghvi/fixprotocoldemo"><b>example</b></a>, rest of all people can take a pain to read the following:<br />
<br />
<u>Some jargons about the FIX protocol:</u><br />
The Financial Information eXchange ("FIX") Protocol is a series of messaging specifications for the electronic communication of trade-related messages. It has been developed through the collaboration of banks, broker-dealers, exchanges, industry utilities and associations, institutional investors, and information technology providers from around the world. These market participants share a vision of a common, global language for the automated trading of financial instruments ( from Fix Protocol Organization).<br />
<b><span lang="FR" style="font-family: Calibri; font-size: 12pt;"></span></b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-DhorQEHlAFQ/TttWOd6LJRI/AAAAAAAAAYY/R4Rkxxpy58Y/s1600/fixProtocol.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-DhorQEHlAFQ/TttWOd6LJRI/AAAAAAAAAYY/R4Rkxxpy58Y/s1600/fixProtocol.jpg" /></a></div>
<br />
<u><br /></u><br />
<u><br /></u><br />
<u><br /></u><br />
<u><br /></u><br />
<u>Some GYAAN:</u><br />
If every exchange in the world communicates "ONLY" with their own developed messages (Proprietary Protocol ) than for every exchange there will be need to develop seperately the trading client infrastructure or trading client platform. This is a painful work as to maintain a client with the proprietary messaging or Proprietary Protocol (Message) for every exchange. And keeping the client in the accordance with the exchange notification(s) for upgrades of messages to communicate is a big problem.<br />
And in that case doing a trade with client which supports multiple exchange at the same time seems far from the reality. Of course a client can manage support for few exchange but a large number of exchange will not be possible ( which is the need in the today's world).<br />
<br />
With FIX based application its easy to develop OMS ( Order Management System) and EMS ( Execution Management System) faster and scalable. It can be used for Algo Trading but its gonig to be bit slower, so for Algorithmic Trading Proprietary Protocol is a good choice.<br />
<br />
<br />
A typical FIX message looks like as:<br />
br />
<br />
<pre><20111204-11:02:59, FIX.4.2:FixClient8019->FixAcceptor, outgoing>(8=FIX.4.2|9=80|35=A|34=14|49=FixClient8019|52=20111204-11:02:59.353|56=FixAcceptor|98=0|108=60|10=219|)
</pre>
where every tag has a specific meaning, in the above tags are 8,9,35,34,49,52 etc.
<br />
<br />
Lets jump to the FIX based Acceptor and Iniator.<br />
<br />
FixAcceptor: Which listens on a specific socket and works as a server. It is the Stock Exchange. Number of clients can connect to it. All the properties for the Acceptor is defined in the acceptor.cfg file.<br />
FixInitiator: Which connects to the FixAcceptor on a specific port defined by Acceptor. It is a trading client. Client can connect with different fix implemenation versions which a client and exchange supports. I am using FIX.4.2 version for the example.<br />
<br />
Explanation about the .cfg and fix message is beyond the scope of this blog. ha ha ha, i saved some time. Dirty trick to escape but i have no choice as this is a vast subject and neither i have time nor the experities to cover everything.<br />
<br />
Create FixAcceptor.java and FixIniator.java either by implementing Application interface or
by extending ApplicationAdapter class.<br />
<br />
First some snippet from Exchange i.e. FixAcceptor.java<br />
<br />
<pre>public class FixAcceptor extends MessageCracker implements Application
</pre>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="http://2.bp.blogspot.com/-oI4PtGNjScg/TttJhGCKR-I/AAAAAAAAAYQ/wZNoMyahozU/s1600/ApplicationInterfaceMethods.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="274" src="http://2.bp.blogspot.com/-oI4PtGNjScg/TttJhGCKR-I/AAAAAAAAAYQ/wZNoMyahozU/s320/ApplicationInterfaceMethods.jpg" width="320" /></a></div>
<br />
<br />
<br />
Start the acceptor based on the configuration defined in acceptor.cfg<br />
<br />
<pre>SessionSettings settings = new SessionSettings("./conf/acceptor.cfg");
FixAcceptor acceptor = new FixAcceptor();
ScreenLogFactory screenLogFactory = new ScreenLogFactory(settings);
DefaultMessageFactory messageFactory = new DefaultMessageFactory();
FileStoreFactory fileStoreFactory = new FileStoreFactory(settings);
SocketAcceptor socketAcceptor = new SocketAcceptor(acceptor,
fileStoreFactory, settings, screenLogFactory,messageFactory);
socketAcceptor.start();
</pre>
<br />
onMessage(...) : there are many overloaded onMessage() methods are defined in the api, for the purpose of example i have overridden <br />
<br />
<pre>@Override
public void onMessage(NewOrderSingle order, SessionID sessionID)
<span class="Apple-tab-span" style="white-space: pre;"> </span>throws FieldNotFound, UnsupportedMessageType, IncorrectTagValue {
...
}
</pre>
exchange checks for the match for the requested order by the client and in case of match it process the order and updates the client with the execution report. Have a look in the example to see the implementation.<br />
<br />
<br />
<br />
Now some snippet from Trading Client i.e. FixIniator.java<br />
Here i am extending ApplicationAdapter instead of implementing Application as its done in FixAcceptor.java
<br />
<br />
<br />
<span class="Apple-style-span" style="color: #333333; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif;"><span class="Apple-style-span" style="font-size: 15px; line-height: 20px;"><br /></span></span><br />
<pre>public class FixInitator extends ApplicationAdapter
</pre>
Start the initiatorr based on the configuration defined in initiator.cfg
<pre>FixInitator fixIniator = new FixInitator();
SessionSettings sessionSettings = new SessionSettings("./conf/initiator.cfg");
ApplicationAdapter application = new FixInitator();
FileStoreFactory fileStoreFactory=new FileStoreFactory(sessionSettings);
ScreenLogFactory screenLogFactory=new ScreenLogFactory(sessionSettings);
DefaultMessageFactory defaultMessageFactory=new DefaultMessageFactory();
fixIniator.socketInitiator = new SocketInitiator(application,
fileStoreFactory, sessionSettings, screenLogFactory,defaultMessageFactory);
fixIniator.socketInitiator.start();
// prepare order NewOrderSingle order = new NewOrderSingle( ...)
try {
// send order to target which is nothing but the exchange/acceptor
Session.sendToTarget(order, sessionID);
} catch (SessionNotFound e) {
e.printStackTrace();
}
</pre>
<br />
I have overridded method fromAdmin(...) where client will receive from the admin where one of the message is the ExecutionReport which is sent by exchange/acceptor.
<br />
<pre>@Override
public void fromApp(Message message, SessionID sessionId)
</pre>
</div>
<br />
COMPLETE EXAMPLE CAN BE DOWNLOADED FROM <a href="https://sites.google.com/site/deepaksinghvi/fixprotocoldemo"><b>HERE</b></a></div>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com4tag:blogger.com,1999:blog-2415001147493991407.post-51045460310923969592011-11-05T21:29:00.003+05:302011-11-05T21:55:24.140+05:30Android App / Mobile Trading Client<div dir="ltr" style="text-align: left;" trbidi="on">
My first android app for trading client.<br />
Very happy today, finally i did it, it was long pending task.<br />
I have intention to use QuickFix/J a Fix Protocol based open source library.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-DaXx4t98Yvw/TrVXUradrlI/AAAAAAAAAYE/vjUrLdHaP3A/s1600/SpeedTradeFixBasedMobileClient.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="452" src="http://3.bp.blogspot.com/-DaXx4t98Yvw/TrVXUradrlI/AAAAAAAAAYE/vjUrLdHaP3A/s640/SpeedTradeFixBasedMobileClient.png" width="640" /></a></div>
<br />
For the android app demo purpose (<a href="https://sites.google.com/site/deepaksinghvi/android_app_trading_client">click here to download sample code</a>) i have removed the JMS web service call which can send the order to Fix Protocol based client and just shown the order<br />
<br />
<br />
<br />
<i> <span class="Apple-style-span" style="color: #134f5c;"><b>JMS Web Service</b></span></i><span class="Apple-style-span" style="color: #134f5c;"> </span> <span class="Apple-style-span" style="color: #134f5c;"><b> Fix Message on TCP/IP To Exchange</b></span> <br />
<span class="Apple-style-span" style="color: purple;">Mobile Client ===> JMS Queue ===> Fix based Client (Broker) ===> Fix based Server (Exchange)</span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b> <i>QuickFix/J based client Parser</i></b></span><br />
<br />
<u>FROM CLIENT TO BROKER-EXCHANGE DATA FLOW</u><br />
<br />
<span class="Apple-style-span" style="color: purple;"><br /></span><br />
<span class="Apple-style-span" style="color: purple;"> </span><b><i><span class="Apple-style-span" style="color: #134f5c;">Order Status Update (Broadcast-Uni-cast/ TCP-IP Message)</span></i></b><br />
<span class="Apple-style-span" style="color: purple;">Fix based Server (Exchange) ===> Fix based Client (Broker) ===> Mobile Client</span><br />
<b style="color: #134f5c;"><i>Order Status Update (may be SMS)</i></b><br />
<b style="color: #134f5c;"><i><br /></i></b><br />
<b style="color: #134f5c;"><i> </i></b><u>FROM EXCHANGE-BROKER TO CLIENT DATA FLOW</u><br />
<br />
<br />
User enters the required Symbol(Script) and other details for Buy/Bid and click on Order button. Android app sends the data to JMS queue which is running on a remote machine at Broker's office and which is based on JMS WebService .<br />
<br />
Broker has a Fix protocol based parser running which collects the order from the queue and prepare the Fix based message to Fix protocol based Server, which is located at Stock Exchange.<br />
<br />
Once order is confirmed from Exchange the same is sent to Broker and which updates the Client.<br />
<br />
Android made the work easy for development, i am not able to find anything much to explain anything in detail. May be have look of the <a href="https://sites.google.com/site/deepaksinghvi/android_app_trading_client">code</a>.<br />
<br />
Note: Target audience for this blog and code is only developer and not the android phone user.</div>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com7tag:blogger.com,1999:blog-2415001147493991407.post-82247604682542875302011-09-14T10:56:00.001+05:302012-03-29T17:01:51.976+05:30Remote Debugging In Eclipse<div dir="ltr" style="text-align: left;" trbidi="on">
Remote debugging capability with different debuggers( IDEs) are nice features for developers. Purpose of this blog is to share this information that how to
use remote debugging tools effectively. Here i tried to give example for eclipse IDE. <br />
<br />
Remote debugging in eclipse can be done in two ways:<br />
a) Socket Attach<br />
b) Socket Listen<br />
<br />
<br />
<u> Socket Attach:</u> For e.g. we want to debug an application which is deployed on tomcat and we want to debug that.
Attach the application process through eclipse once we start our tomcat in debug mode.<br />
<br />
<span class="Apple-style-span" style="color: #e06666;"><b><i>Step 1:</i></b></span><br />
How to start tomcat in debug mode:<br />
<b>$./catalina.sh jpda start </b>(in case of unix based machines) or<br />
<b>> catalina.bat jpda start</b> ( in case of windows based machines)<br />
<br />
This is going to start a tomcat in debug mode and a socket with port as 8000 can be attached to it.<br />
In case if user wants to attach to different port than following can be done.<br />
<b>export JPDA_ADDRESS=8000 </b><b> </b>(in case of unix based machines)<br />
<b><span class="Apple-style-span" style="font-weight: normal;"><b>$./catalina.sh jpda start</b></span></b><br />
<b><span class="Apple-style-span" style="font-weight: normal;"><b><br /></b></span></b><br />
<b>set JPDA_ADDRESS=8000</b> (in case of windows based machines)<br />
<b><span class="Apple-style-span" style="font-weight: normal;"><b>> catalina.bat jpda start</b></span></b><br />
<br />
<span class="Apple-style-span" style="color: #e06666;"><b><i>Step 2:</i></b></span><br />
Open debug configuration on eclipse and add new Remote Java Application.<br />
Connection Type should be Standard (Socket Attach).
Give a host name or ip in the value for Host in connection properties where the remote process is running.<br />
For e.g.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-v98IxNVQrD8/TnAtXDHCQII/AAAAAAAAAXw/1tO6aZSbxyY/s1600/RemoteDebugging_SocketAttach.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="256" src="http://1.bp.blogspot.com/-v98IxNVQrD8/TnAtXDHCQII/AAAAAAAAAXw/1tO6aZSbxyY/s320/RemoteDebugging_SocketAttach.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
For Other application server like WebSphere:<br />
Open admin console and open Servers->Server Type -> WebSphere Application Servers.<br />
Than choose your server from the list of available servers. And than select Debugging service.<br />
for e.g:<br />
<span class="Apple-style-span" style="font-weight: bold;">Application
servers > server1
> Debugging service</span><br />
Make check box "Enable service at server startup" as selected.<br />
default port for debug is 7777, change it if you want to debug on different port.<br />
Restart the server and attach your debugger ( IDE or eclipse) to it.<br />
<br />
<br />
<u> Socket Listen:</u> When we want to start a listener first and the remote application is required to be attached to the eclipse <br />
debugger, in that case. First we need to start the eclipse debugger in Socket Listten mode and attach the application.<br />
<br />
<span class="Apple-style-span" style="color: magenta;"><b><i>Step 1:</i></b></span><br />
Open debug configuration in eclipse and add new Remote Java Application.<br />
Connection Type should be Standard (Socket Listen).<br />
Port in connection properties should be 8000 if you want remote application should connect using this port value.<br />
For e.g. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-JqxKIozFuhU/TnAvfFIeouI/AAAAAAAAAX4/PSr7mshRRic/s1600/RemoteDebugging_SocketListen.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="256" src="http://1.bp.blogspot.com/-JqxKIozFuhU/TnAvfFIeouI/AAAAAAAAAX4/PSr7mshRRic/s320/RemoteDebugging_SocketListen.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<span class="Apple-style-span" style="color: magenta;"><b><i>Step 2: </i></b></span><br />
<b><span class="Apple-style-span" style="color: #bf9000;">Options a)</span></b> if wants to debug with ant than in your build.xml add<br />
<b><jvmarg line="-Xdebug -Xrunjdwp:transport=dt_socket,address=10.240.1.14:8000,server=n" /></b>
for e.g.<br />
<pre><target name="execute" description="Test">
<java classname="${main.class}" fork="true" failonerror="true">
<classpath refid="class.path" />
<arg value="${myargument}" />
<jvmarg value="${myjvmargument}" />
<jvmarg value="-Xrs" />
<jvmarg line="-Xdebug -Xrunjdwp:transport=dt_socket,address=10.240.1.14:8000,server=n" />
</java>
</target>
</pre>
<span class="Apple-style-span" style="font-family: 'Times New Roman'; white-space: normal;"><span class="Apple-style-span" style="color: #bf9000;"><b>Option b)</b></span> If you wants to debug with .bat file or on command prompt</span>
<b><br /></b><br />
<b>Java -Xdebug -Xrunjdwp:transport=dt_socket,address=10.240.1.14:8000 -jar MyJar.jar</b><br />
<br />
<br />
<span class="Apple-style-span" style="color: #bf9000;"><b>Options c) </b></span>if you wants to debug a application which is again an eclipse ( or eclipse based plugins) than you can start the eclipse application as<br />
<b><br /></b><br />
<b>eclipse.exe -vmargs -Dosgi.requiredJavaVersion=1.6 -Xms256m -Xmx512m -Xdebug -Xrunjdwp:transport=dt_socket,address=10.240.1.14:8000,suspend=n</b><br />
<br />
10.240.1.14 IP where you want to debug and where your debugger is running.<br />
Debugging in JBoss can be debugged by modifying JAVA_OPTS in the file which is used to start the server in the similar way as mentioned above.<br />
IBM WebSphere is already explained above.</div>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-11298155759355362562011-06-24T09:45:00.003+05:302011-09-14T10:59:16.778+05:30Dynamic Proxy for gui component listener using InvocationHandler.Listeners for gui is always easy to create and difficult to maintain.<br />
As they will create additional classes for that and in case we need to <br />
give a patch or something it can be a real pain.<br />
<br />
Lets say that a Frame (MyFrame.java) has 10 components and each of them has a listener than there will be 10 additional classes ( MyFrame$1.class,... , MyFrame$10.class, etc).<br />
Now if a fix is given for one of the component and we really do not know which one to give as a hot fix, we will be ending up in giving all the 11 classes.<br />
Another thing which might also force to give all the classes if we change the serialversionId.<br />
<br />
So how to overcome this, i have tested it and found it works well. <br />
But is it the right thing to do not sure but good to know.<br />
<br />
We can use the InvocationHandler to handle this situation which can call the required method of the frame where we can have the logic/code to be run using Dynamic Proxies to call whenever event is raised.<br />
<br />
<br />
<pre>import java.lang.reflect.*;
/**
* @author deepak.singhvi
*
*/
public class InvocationHandlerProxy implements
InvocationHandler {
private Object obj;
public static Object newInstance(Object obj) {
return Proxy.newProxyInstance(obj.getClass()
.getClassLoader(), obj.getClass().getInterfaces(),
new InvocationHandlerProxy(obj));
}
public InvocationHandlerProxy(Object obj) {
this.obj = obj;
}
public Object invoke(Object proxy, Method m, Object[] args)
throws Throwable {
Object result;
try {
result = m.invoke(obj, args);
} catch (InvocationTargetException e) {
throw e.getTargetException();
} catch (Exception e) {
throw new RuntimeException("unexpected invocation exception: "
+ e.getMessage());
}
return result;
}
}
</pre><br />
For every gui component, for e.g. button instead of overriding actionListener for some action event we can call.<br />
<br />
<pre>ActionListener foo = (ActionListener) InvocationHandlerProxy.newInstance(new FooWithoutArgActionListener(this,"methodWithoutArgument",new Class[0]));
</pre><br />
Listener class would implement ActionListener as usual but the method to be called when a<br />
specific event is raised will be defined in the same class where listener is created, without creating lister anonymously.<br />
<br />
<pre>public class FooWithoutArgActionListener
implements ActionListener {
Method method;
Object parent;
public FooWithoutArgActionListener(Object classObject,
String methodName, Class<?>[] parameterTypes ){
method = getTargetMethod(classObject, methodName, parameterTypes);
this.parent = classObject;
}
@Override
public void actionPerformed(ActionEvent e) {
....
method.invoke(parent, new Object[0]);
...
}
private Method getTargetMethod(Object target,
String methodName, Class<?>[] parameter)
{
// find the target method which should get
// executed when actionPerformed is called.
}
</pre><br />
<br />
<font size="5" face="arial" ><b><a href="https://sites.google.com/site/deepaksinghvi/dynamicproxylistenr" style="color: #FF00FF" target=_blank>Click here to get the Complete example</a></b></font>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-73033203986579128192011-06-04T11:53:00.001+05:302011-09-21T10:30:21.824+05:30Find from which jar a class was loaded<div dir="ltr" style="text-align: left;" trbidi="on">
In order to find at runtime where from file system a class was loaded, following command would be useful:<br />
<br />
System.out.println(<myjavaclassname>.class.getProtectionDomain().getCodeSource().getLocation());<br />
<br />
Output will be something like this:<br />
file:/C:/TestWorkSpace/lib/MyLibrary.jar<br />
<br />
It can be helpful while debugging some class loading problems.</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-70516041037025151122011-03-11T09:52:00.016+05:302011-09-14T10:59:38.186+05:30Hibernate and DAOUNDER CONSTRUCTION<br />
<br />
<br />
<pre style="font: 8pt courier" >package com.dao;
public interface DAOIntf<T> {
public void update(T obj);
public void save(T obj);
public void delete(T obj);
}
</pre><br />
<br />
<br />
<br />
<br />
<pre style="font: 8pt courier">package com.dao;
import org.hibernate.HibernateException;
import org.hibernate.Session;
import org.hibernate.Transaction;
import com.util.HibernateUtil;
public class AbstractDAOImpl<T> implements DAOIntf<T>{
@Override
public void update(T obj) {
save(obj);
}
@Override
public void save(T obj) {
Session session = HibernateUtil.getSessionFactory().openSession();
Transaction transaction = null;
try {
transaction = session.beginTransaction();
session.save(obj);
transaction.commit();
} catch (HibernateException e) {
transaction.rollback();
e.printStackTrace();
} finally {
session.close();
}
}
@Override
public void delete(T obj) {
Session session = HibernateUtil.getSessionFactory().openSession();
Transaction transaction = null;
try {
transaction = session.beginTransaction();
session.delete(obj);
transaction.commit();
} catch (HibernateException e) {
transaction.rollback();
e.printStackTrace();
} finally {
session.close();
}
}
}
</pre><br />
<br />
<br />
ProductDAOIntf.java<br />
<pre style="font: 8pt courier">package com.dao;
import java.util.List;
public interface ProductDAOIntf<T> extends DAOIntf<T> {
public T getProductByID(int id);
public List<T> getAllProducts();
}
</pre><br />
<br />
ProductDAOImpl.java<br />
<pre style="font: 8pt courier">package com.dao;
import java.util.List;
import org.hibernate.HibernateException;
import org.hibernate.Session;
import com.hbm.Product;
import com.util.HibernateUtil;
public class ProductDAOImpl<T> extends AbstractDAOImpl<T> implements ProductDAOIntf<T> {
@Override
public T getProductByID(int id) {
Session session = HibernateUtil.getSessionFactory().openSession();
T product=null;
try {
product =(T) session.get(Product.class,id);
} catch (HibernateException e) {
e.printStackTrace();
} finally {
session.close();
}
return product;
}
@Override
public List<T> getAllProducts(){
Session session = HibernateUtil.getSessionFactory().openSession();
List<T> list = null;
try {
list = session.createQuery("from Product").list();
} catch (HibernateException e) {
e.printStackTrace();
} finally {
session.close();
}
return list;
}
}
</pre><br />
Product.java<br />
<pre style="font: 8pt courier">package com.hbm;
import java.io.Serializable;
public class Product implements Serializable {
public Integer getProduct_Id() {
return Product_Id;
}
public void setProduct_Id(Integer product_Id) {
Product_Id = product_Id;
}
public String getProduct_Name() {
return Product_Name;
}
public void setProduct_Name(String product_Name) {
Product_Name = product_Name;
}
public Double getProduct_Price() {
return Product_Price;
}
public void setProduct_Price(Double product_Price) {
Product_Price = product_Price;
}
public Integer getProduct_QOH() {
return Product_QOH;
}
public void setProduct_QOH(Integer product_QOH) {
Product_QOH = product_QOH;
}
Integer Product_Id;
String Product_Name;
Double Product_Price;
Integer Product_QOH;
@Override
public String toString() {
return "Id : " + Product_Id + "\t Name " + Product_Name + "\tQOH " + Product_QOH;
}
}
</pre><br />
Product.hbm.xml<br />
<pre style="font: 8pt courier"><?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping>
<class name="com.hbm.Product" table="product" >
<id name="Product_Id" type="int"> <generator class="identity"></generator></id>
<property name="Product_Name" type="string" length="50" />
<property name="Product_Price" type="double" lazy="true" />
<property name="Product_QOH" type="int"/>
</class>
</hibernate-mapping>
</pre>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com0tag:blogger.com,1999:blog-2415001147493991407.post-38887661751996047092010-12-14T14:05:00.001+05:302011-12-04T23:37:38.837+05:30Thread Working For Fixed Time Even In Case Of Exceptions (UncaughtExceptionHandler)Somebody asked me how to keep running a thread for a fixed interval even there is some exception.<br />
At that point of time i had no answer.<br />
<br />
But now i think, i have the answer.<br />
<br />
And this can be done using the interface Thread.UncaughtExceptionHandler.<br />
<br />
UncaughtExceptionHandler is an inner interface of Thread class.<br />
<br />
We can implement it and set it using the method setUncaughtExceptionHandler() of Thread class.<br />
<br />
Here is an example.<br />
<br />
<pre>class MyThread extends Thread {
private boolean flag;
protected int counter=0;
MyThread(int counterInitialValue) {
this.counter = 0;
}
public MyThread(Runnable r,int counterInitialValue) {
super(r);
this.counter = counterInitialValue;
this.flag = true;
}
@Override
public void run() {
if (!flag) {
flag = true;
System.out.println("Thread Exec Started");
} else {
System.out.println("Thread Running Again");
}
while (counter <20) {
counter++;
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
// do some work here....
if(counter % 3 == 0){
System.out.println(" Thread working ,
counter value is : "+ counter);
}
if (counter == 10) {
throw new RuntimeException("I am throwing exception");
}
}
}
}
class MyThreadExceptionHandler implements
Thread.UncaughtExceptionHandler {
@Override
public void uncaughtException(Thread t,
Throwable e) {
MyThread oldThread = (MyThread) t;
System.out.println("Received exception "+
e.getMessage());
if (oldThread.counter < 20)
{
System.out.println("Value of counter
(restart required) : " + oldThread.counter);
MyThread newThread = new MyThread(oldThread,11);
newThread.start();
}
else
{
System.out.println("Thread work done");
System.out.println("Value of counter : " +
oldThread.counter);
}
}
}
public class ThreadRunningForFixedTime {
public static void main(String[] args) {
MyThread th = new MyThread(0);
MyThreadExceptionHandler handler =
new MyThreadExceptionHandler();
th.setUncaughtExceptionHandler(handler);
th.start();
}
}
</pre>Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com1tag:blogger.com,1999:blog-2415001147493991407.post-29724343707519695082010-12-06T17:34:00.003+05:302013-10-20T01:29:13.558+05:30How to make a non-serializable class serializable<div dir="ltr" style="text-align: left;" trbidi="on">
I have a class which is not serializable. If i will try to serialize the object of it than i am going to get exception "java.io.NotSerializableException"<br />
<br />
And now if this class is not not modifiable/available to modify (may be its a third party), and so we do not have any control.<br />
<br />
for e.g following is the non serializable class<br />
<pre>class MyNonSerializableClass {
String str = new String("mystring");
MyNonSerializableClass(){
}
MyNonSerializableClass(String s) {
this.str = s;
}
public String getStr() {
return str;
}
public void setStr(String str) {
this.str = str;
}
}
</pre>
<br />
<br />
<br />
In order to serialize the object we can write a wrapper class over MyNonSerializableClass and which is implementing Serializable interface. <br />
And the most important thing, we control the readObject and writeObject methods so that we can decide what need to be written with the wrapper class.<br />
<br />
something like this:<br />
<br />
<pre>class MyNonSerializableClassWrapper extends MyNonSerializableClass implements Serializable {
MyNonSerializableClassWrapper(){
super();
}
MyNonSerializableClassWrapper(String s) {
super(s);
}
private void writeObject(ObjectOutputStream out) throws IOException {
// not required the default write object
// ----> out.defaultWriteObject();
out.writeObject(super.getStr());
}
private void readObject(ObjectInputStream in) throws IOException {
// not required the default read object
// ----> in.defaultReadObject();
try {
super.setStr((String) in.readObject());
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
}
</pre>
<br />
<br />
As the wrapper class itself does not have any properties/attributes so defaultread and write is not required.<br />
By getting the super class value and setting it during the write and read to serialized object we can achieve the serialization and deserialization even if a class is not implementing Serializable interface.<br />
<br />
<br />
<pre>private void WriteObjectToFile() {
try {
FileOutputStream fo = new FileOutputStream("c:\\test.ser");
ObjectOutputStream os = new ObjectOutputStream(fo);
os.writeObject(new MyNonSerializableClassWrapper("This is a test of
serialization"));
fo.close();
}
catch (FileNotFoundException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
}
private void ReadObjectFromFile() {
try {
FileInputStream fi = new FileInputStream("c:\\test.ser");
ObjectInputStream in = new ObjectInputStream(fi);
MyNonSerializableClassWrapper mw = (MyNonSerializableClassWrapper)
in.readObject();
fi.close();
System.out.println("serialized object value is: " + mw.getStr());
}
catch (FileNotFoundException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
</pre>
</div>
Deepak Singhvihttp://www.blogger.com/profile/04857569060749773854noreply@blogger.com3