#Apress “Practical Enterprise Data Lake Insights” – Published!

Hello All,

Gives me immense pleasure to announce the release of our book “Practical Enterprise Data Lake Insights” with Apress. The book takes an end-to-end solution approach in a data lake environment that includes data capture, processing, security, and availability. Credits to the co-author of the book, Venkata Giri and technical reviewer, Sai Sundar.

The book is now available at various channels as subscription and in print (on request!) and e-book (e.g., Amazon/Kindle, Barnes & Noble/nook, Apress.com). Below are the Apress and Amazon links –

Apress – https://www.apress.com/gb/book/9781484235218

Amazon – https://www.amazon.com/Practical-Enterprise-Data-Lake-Insights/dp/1484235215/

Thank you for all your confidence, support, and encouragement. Thanks Monica Caldas, CIO (GE Transportation) for helping us with the Foreword column.

Brief about the book –

When designing an enterprise data lake you often hit a roadblock when you must leave the comfort of the relational world and learn the nuances of handling non-relational data. Starting from sourcing data into the Hadoop ecosystem, you will go through stages that can bring up tough questions such as data processing, data querying, and security. Concepts such as change data capture and data streaming are covered. The book takes an end-to-end solution approach in a data lake environment that includes data security, high availability, data processing, data streaming, and more.

Each chapter includes application of a concept, code snippets, and use case demonstrations to provide you with a practical approach. You will learn the concept, scope, application, and starting point. Use this practical guide to successfully handle the challenges encountered when designing an enterprise data lake and learn industry best practices to resolve issues.

 

What You’ll Learn:

  • Get to know data lake architecture and design principles
  • Implement data capture and streaming strategies
  • Implement data processing strategies in Hadoop
  • Understand the data lake security framework and availability model

Grab your copies fast. Enjoy reading!

Saurabh

Advertisements

Harness the Power of Data in a Big Data Lake

Last year November, I got the opportunity to present at AIOUG Sangam, 2017. My session was titled as “Harness the Power of Data in a Big Data Lake”. The abstract is as below –

Data lake is relatively a new term when compared to all fancy ones since the industry realized the potential of data. Industry is planning their way out to adopt big data lake as the key data store but what challenges them is the traditional approach. Traditional approaches pertaining to data pipelines, data processing, data security still hold good but architects do need to leap an extra mile while designing big data lake.

This session will focus on this shift in approaches. We will explore what are the road blockers while setting up a data lake and how to size the key milestones. Health and efficiency of a data lake largely depends on two factors – data ingestion and data processing. Attend this session to learn key practices of data ingestion under different circumstances. Data processing for variety of scenarios will be covered as well.

Here is the link to my presentation –

Sangam17_DataInDataLake

The session was an excerpt from my upcoming book on Enterprise Data Lake. The book should be out within a month from now and is available at all online bookstores.

Amazon – https://www.amazon.com/Practical-Enterprise-Data-Lake-Insights/dp/1484235215/

When designing an enterprise data lake you often hit a roadblock when you must leave the comfort of the relational world and learn the nuances of handling non-relational data. Starting from sourcing data into the Hadoop ecosystem, you will go through stages that can bring up tough questions such as data processing, data querying, and security. Concepts such as change data capture and data streaming are covered. The book takes an end-to-end solution approach in a data lake environment that includes data security, high availability, data processing, data streaming, and more.
Each chapter includes application of a concept, code snippets, and use case demonstrations to provide you with a practical approach. You will learn the concept, scope, application, and starting point.

What You’ll find in the book

  • Get to know data lake architecture and design principles
  • Implement data capture and streaming strategies
  • Implement data processing strategies in Hadoop
  • Understand the data lake security framework and availability model

Enjoy reading!

Saurabh

My session at #AIOUG #Sangam16 – Transition from Oracle DBA to Big Data Architect

Big thanks to all those who turned up for my early morning session on Saturday Nov 12th, 2016. I know it was a tough call after a week’s work but thanks for making the right decision. A full-house is an extreme delight for a speaker.

You can download the session deck either from the Sangam website or from the link below.

Sangam16_TransformIntoBigDataArchitect

I hope the session was useful to all. If you have any doubts or comments, feel free to comment below. If you have a feedback on the session, I would surely love to hear.

I love to feature in AIOUG conferences and events; Sangam being one of them. In addition to attending session, we get a chance to meet and greet the geeks and techies from around the world. I must confess that I get to meet many of them only at events like this. While I was fortunate to meet Arup Nanda, Syed Jaffer Hussain, Aman, Nassyam, Sai, Satyendra, Kuassi Mensah, I had a pleasure of spending time with Oracle colleagues and many others during these two days.

Sangam 2016 was huge; it continues to grow by a fold. 100+ session in two days and distinguished speakers from all over the world. Thanks to AIOUG team and volunteers who coordinated and managed the event fairly well.

Thanks again!

Query materialized view refresh timestamp

Unlike my lengthy posts, this is really a quick one.

Alright, so how do you get to know when your materialized view got refreshed? Well, no biggies. There are bunch of dictionary views who capture refresh date but none of them give you timestamp. For fast refresh, you can work with SCN or timestamp based MView logs but for complete refresh M-views, this can be tricky and here is a quick easy solution to retrieve timestamp information.

You can query ALL_MVIEW_ANALYSIS dictionary view that captures System change number (SCN) of the last refresh operation (i.e start refresh time). Use SCN_TO_TIMESTAMP function to translate SCN into timestamp. Here is the query –

SELECT owner,
mview_name,
last_refresh_scn,
SCN_TO_TIMESTAMP (last_refresh_scn) refresh_timestamp
FROM all_mview_analysis
WHERE mview_name = <>;

Try it yourself. I recommend this dictionary table as it also lets you know the time taken in fast or full refresh (FULLREFRESHTIM/INCREFRESHTIM). Don’t miss this nice article “How long did Oracle materialized view refresh run?” by Ittichai C.

I/O Resource Management on Exadata

Consolidation is a key enabler for Oracle database deployments on both public and private clouds. Consolidation reduces the overall IT costs by optimizing the operational and capital expenses. In addition, it enhances the effective utilization of cloud resources. The Exadata database machine has been optimized to run schemas and databases with mixed workloads together, making it the best platform for consolidation.

Organizations follow different approaches to consolidate database instances. Some of the prominent approaches of consolidation are virtualization, schema based consolidation and database aggregation on a single high end physical server. Oracle Database 12c introduces Multitenant Architecture to allow secure consolidation of databases on cloud and achieve benefits like tenant isolation, manage many-as-one capability and enhanced customer satisfaction.

For effective database consolidation, Exadata makes use of Oracle resource management (database resource management, network resource management and I/O resource management). The Exadata IORM enhances the stability of mission critical applications and ensures availability of all databases which share the server resources. The I/O resource plan provides the framework for queuing the low-priority requests while issuing high-priority requests. This post will focus on configuring, enabling and monitoring IORM plans on Exadata database machines.

Oracle Database Resource Manager

On a database server, the resources are allocated by the operating system, which may be inappropriate and inefficient in maintaining database health. The server stability and database instance is impacted by the high CPU load, thus resulting in sub-optimal performance of the database. Oracle Database Resource Manager, first introduced in Oracle Database 8i, can help you by governing the allocation of resources to the database instance and assuring efficient utilization of CPU resources on the server. It is a database module which allocates the resources to a resource consumer group as per a set of plan directives in a fair way. A resource consumer group comprises of database sessions with “like” resource requirements.

 A resource plan can manage the allocation of CPU, disk I/Os and parallel servers among schemas in a single database or multiple databases in a consolidated environment. An intra-database plan can be created to manage allocation across multiple schemas or services within a single database. On an Exadata database machine, the disk I/Os can be managed across multiple databases using IO Resource Manager (IORM) or inter-database plan. The Oracle Database Resource Manager is a feature of Oracle Database Enterprise Edition. However, starting with Oracle Database 11g, it can also be used in Standard Edition to manage maintenance tasks through a default maintenance plan.

This post focuses on configuration of IORM plans on the Exadata database machine. In the post, we shall discuss how to manage disk I/Os, manage flash usage, manage standby database using IORM.

Read the complete post at Community.Oracle here – https://community.oracle.com/docs/DOC-998939

#AIOUG #OTNYathra – A note of thanks

Folks,

OTNYathra, the OTN tour organized by AIOUG got over last week. Thanks to the Team AIOUG for putting in loads of efforts to make this event a grand success. A round of applause to Sai, Hari, and Veera for coordinating it so well.

This was my second OTNYathra and I could easily sense the penetration AIOUG has made in last couple of years. Huge response, active participation, and willingness to volunteer. AIOUG is growing as a family. Attendees find such events a great stage to talk, share, and expand professional network.

The cities that I covered this year were Bangalore, Hyderabad, and Mumbai. I presented on Oracle BigData SQL and Oracle Backup Cloud Service. You can download the session decks from the below links –

AIOUG_BigDataSQL

AIOUG_BackupCloudService

Thanks to all those who turned up for my sessions. I’m extremely satisfied by the response and level of interest. I felt truly respected while awarding the complimentary copy of my book to the winners of a surprise quiz during the BDS session. I’m sure you guys will like it as I’ll wait for your feedback/comments.

During the tour, I was glad to share the stage with Riyaj Shamsudeen, Biju Thomas, Connor McDonald, Satyendra, Chetan Vithlani, Aman Sharma, Vivek Sharma, Nassyam Basha, Chaitanya, Sai, Hari, and Veera. It was a delight to see many faces coming out of social handles and meeting them in person.

Thanks again. Stay in touch!

Saurabh (@saurabhkg)

 

Attend my session at #AIOUG OTN Yathra and win a complimentary copy of my book

giveaway

 

Otnyatra

Are you passionate to learn Advanced PL/SQL? Would you like to know the new development features in Oracle #DB12c? Are you pursuing Oracle Developer certification exam (1Z0-146)? If yes, then this post is for you.

Announcement

I’m pleased to announce that I would be giving away free complimentary copies of my latest book during the coming OTN Yathra. In addition, you can avail 10% discount on the e-book. Use the code **TradeWnT10** while checking out the book from Packt’s page. The discount code is valid from April 17th to May 3rd, 2016.

Just so you know, I would be speaking in Bangalore, Hyderabad, and Mumbai. To know more about the sessions, check my previous blog post.

How to participate and engage? 

Attend my session and participate in the quiz during the session. The winner walks away with the free paperback copy of the book. I would really appreciate if the winners can post their reviews on their blogs, social handles or Amazon.

Book info – Want to know what is this book all about? It’s here.

Book reviews – Want to check out the reviews on this book? It’s here.

Thanks! I would love to see you comments and feedback on this post as well as here.