Black Friday Deal : Up to 40% OFF! + 2 free self-paced courses + Free Ebook  - SCHEDULE CALL

- Hadoop Blogs -

Hadoop Hive Modules & Data Type with Examples

Hadoop is an open source framework from Apache. Hadoop is used to analyze huge data volume and store processes. The language used in Hadoop is written in Java and is not an online analytical process, which is used for batch/offline processing. Hadoop is widely in a trend and is being used by Facebook, Yahoo, Google, Twitter, LinkedIn and many more. Hadoop can be scaled up simply by adding nodes in the cluster.

Modules of Hadoop

  1. Hadoop Distributed File System: HDFS was developed by Google. According to HDFS the files will be broken into blocks and stored in nodes over the distributed architecture.
  2. Yarn: In order to manage the cluster, another Resource Negotiator is used which also performs job scheduling.
  3. Map Reduce: Map Reduce helps in reducing the framework which enables the parallel computation for the Java programs for the data using the key-value pair. The Map task helps to input the data and converts it into the data set that can be computed in Key value pair. The Map task output is consumed by reducing the task and then the output of the reducer gives the result that is desired.
  4. Hadoop Common: Hadoop common are Java libraries which are used to start Hadoop and are also used by other Hadoop modules.

Advantages of Hadoop

  • Speedy: In HDFS the data is distributed over the cluster and as they are mapped it helps in faster retrieval of the data. Even the tools to process the data are often on the same servers, thus reducing the processing time.
  • Scalable: Hadoop cluster can be extended by just adding nodes in the cluster.
  • Cost Effective: Hadoop is an open source. As to store data it uses commodity hardware becomes so much more cost-effective when compared to traditional relational database management system.
  • Resistance to failure: HDFS has a unique property using which it can replicate data over the network. When network failures occur or when a node is down, then Hadoop acts efficiently by training the other copy of data and makes use of it. Though the data is replicated three times, but the replication factor is configurable.

Hadoop Hive Modules & Data Type with Examples

Data Types in Hive

In the hive tables, Data types are used for specifying the column/field type. Hive data types can be classified into following categories: All the data types in the Hive are classified into types, given as follows:

1). Primitive Data type Primitive Data Types also divide into 4 types which are as follows:

A). Numeric Data Type The Hive Numeric Data types also classified into two types-

B). Integral Data Types The Hive Integral data types are as follows- TINYINT (1-byte (8 bit) signed integer, from -128 to 127) SMALLINT (2-byte (16 bit) signed integer, from -32, 768 to 32, 767) INT (4-byte (32-bit) signed integer, from –2,147,483,648to 2,147,483,647) BIGINT (8-byte (64-bit) signed integer, from –9,223,372,036,854,775,808 to 9,223,372,036,854,775,807)

C). Floating Data Types The Hive Floating data types are as follows- FLOAT (4-byte (32-bit) single-precision floating-point number) DOUBLE (8-byte (64-bit) double-precision floating-point number) DECIMAL (Arbitrary-precision signed decimal number)

i). Date/Time Data Type The second category of Apache Hive primitive data type is Date/Time data types. The following data types come into this category-

Read: Apache Storm Interview Questions and Answers: Fresher & Experience
  • TIMESTAMP (Timestamp with nanosecond precision)
  • DATE (date)
  • INTERVAL

ii). String Data Type String data types are the third category under Hive data types. Below are the data types:- STRING (Unbounded variable-length character string) VARCHAR (Variable-length character string) CHAR (Fixed-length character string)

iii). Miscellaneous Data Type The two data types come from Hive miscellaneous data types- BOOLEAN (True/false value) BINARY (Byte array)

2). Complex Data Type Following are the complex data types:

  • ARRAY

An Array is the ordered collection of fields. All the fields must be of the same type. Syntax: ARRAY<data_type> E.g. array (1, 2)

  • MAP

A Map is the unordered collection of key-value pairs. Key values can be of any type. Syntax: MAP<primitive_type, data_type> E.g. map(‘a', 1, ‘b', 2).

  • STRUCT

A Struct is the collection of named fields. The fields may be of different types. Syntax: STRUCT<col_name : data_type [COMMENT col_comment],…..> E.g. struct(‘a', 1 1.0),[b] named_struct(‘col1', ‘a', ‘col2', 1,  ‘col3', 1.0)

  • UNION

A union is the value that may be one of a number of defined data. The value is tagged with an integer (zero-indexed) representing its data type in the union. Syntax: UNIONTYPE<data_type, data_type, …> E.g. create_union(1, ‘a', 63)

Read: What Is Hadoop 3? What's New Features in Hadoop 3.0

3). Column Type

  • Integral Type

Following are the 4 data types of integral type: TINYINT, Ex. 100Y SMALLINT, Ex. 100S INT/INTEGER BIGINT, Ex. 100L

  • Strings

The string can be represented with either single quotes (‘) or double quotes (").Hive uses C-style escaping within the strings.

  • Time stamp

The traditional UNIX timestamp is supported in Hive with operational nanosecond precision. Timestamps of text files use format "YYYY-MM-DD HH:MM:SS.fffffffff" and "yyyy-mm-dd hh:mm:ss.ffffffffff".

  • Dates DATE values are described in a particular year/month/day (YYYY-MM-DD) format. E.g. DATE ‘2017-­01-­01’.
  • Decimals

Hive DECIMAL type is similar to a Big Decimal format of Java that represents the  arbitrary precision. The syntax and example are below: “Apache Hive 0.11 and 0.12 has the precision of the DECIMAL type fixed. And it’s limited to 38 digits. Apache Hive 0.13 users can specify the scale and precision when creating tables with the DECIMAL data type using DECIMAL (precision, scale) syntax.  If the scale is not specified, then it defaults to 0 (no fractional digits). If no precision is specified, then it defaults to 10. CREATE TABLE foo ( a DECIMAL, -- Defaults to decimal(10,0)b DECIMAL(9, 7) b DECIMAL(9, 7) )

  • Union Types

Heterogeneous data types collection. “By using create union, we can create an instance.” The syntax and example are as below: CREATE TABLE union_test(foo UNIONTYPE<int, double, array<string>, struct<a:int,b:string>>); SELECT foo FROM union_test; {0:1} {1:2.0} {2:["three","four"]} {3:{"a":5,"b":"five"}} {2:["six","seven"]} {3:{"a":8,"b":"eight"}} {0:9} {1:10.0}

  4). Literals In Hive following literals are used:

Read: What Is The Working Philosophy Behind Hadoop MapReduce?
  • Floating Point Types

These are nothing but numbers with decimal points. This type of data is composed of the DOUBLE data type.

  • Decimal Type

This type is nothing but floating point value with higher range than the DOUBLE data type. The decimal type range is approximate -10-308 to 10308.

5). Null Value In Hive, missing values are represented by the special value NULL.

Conclusion

In this blog on Hive data types we have discussed all the data types in detail with examples. It will definitely provide you a deeper understanding and will help you to understand all the data types in hive easily.



fbicons FaceBook twitterTwitter lingedinLinkedIn pinterest Pinterest emailEmail

     Logo

    JanBask Training

    A dynamic, highly professional, and a global online training course provider committed to propelling the next generation of technology learners with a whole new way of training experience.


  • fb-15
  • twitter-15
  • linkedin-15

Comments

Trending Courses

Cyber Security Course

Cyber Security

  • Introduction to cybersecurity
  • Cryptography and Secure Communication 
  • Cloud Computing Architectural Framework
  • Security Architectures and Models
Cyber Security Course

Upcoming Class

0 day 22 Nov 2024

QA Course

QA

  • Introduction and Software Testing
  • Software Test Life Cycle
  • Automation Testing and API Testing
  • Selenium framework development using Testing
QA Course

Upcoming Class

1 day 23 Nov 2024

Salesforce Course

Salesforce

  • Salesforce Configuration Introduction
  • Security & Automation Process
  • Sales & Service Cloud
  • Apex Programming, SOQL & SOSL
Salesforce Course

Upcoming Class

0 day 22 Nov 2024

Business Analyst Course

Business Analyst

  • BA & Stakeholders Overview
  • BPMN, Requirement Elicitation
  • BA Tools & Design Documents
  • Enterprise Analysis, Agile & Scrum
Business Analyst Course

Upcoming Class

0 day 22 Nov 2024

MS SQL Server Course

MS SQL Server

  • Introduction & Database Query
  • Programming, Indexes & System Functions
  • SSIS Package Development Procedures
  • SSRS Report Design
MS SQL Server Course

Upcoming Class

1 day 23 Nov 2024

Data Science Course

Data Science

  • Data Science Introduction
  • Hadoop and Spark Overview
  • Python & Intro to R Programming
  • Machine Learning
Data Science Course

Upcoming Class

0 day 22 Nov 2024

DevOps Course

DevOps

  • Intro to DevOps
  • GIT and Maven
  • Jenkins & Ansible
  • Docker and Cloud Computing
DevOps Course

Upcoming Class

5 days 27 Nov 2024

Hadoop Course

Hadoop

  • Architecture, HDFS & MapReduce
  • Unix Shell & Apache Pig Installation
  • HIVE Installation & User-Defined Functions
  • SQOOP & Hbase Installation
Hadoop Course

Upcoming Class

0 day 22 Nov 2024

Python Course

Python

  • Features of Python
  • Python Editors and IDEs
  • Data types and Variables
  • Python File Operation
Python Course

Upcoming Class

8 days 30 Nov 2024

Artificial Intelligence Course

Artificial Intelligence

  • Components of AI
  • Categories of Machine Learning
  • Recurrent Neural Networks
  • Recurrent Neural Networks
Artificial Intelligence Course

Upcoming Class

1 day 23 Nov 2024

Machine Learning Course

Machine Learning

  • Introduction to Machine Learning & Python
  • Machine Learning: Supervised Learning
  • Machine Learning: Unsupervised Learning
Machine Learning Course

Upcoming Class

35 days 27 Dec 2024

 Tableau Course

Tableau

  • Introduction to Tableau Desktop
  • Data Transformation Methods
  • Configuring tableau server
  • Integration with R & Hadoop
 Tableau Course

Upcoming Class

0 day 22 Nov 2024

Search Posts

Reset

Receive Latest Materials and Offers on Hadoop Course

Interviews