4.3
星
173 个评分

Karthik Muthuraman 另外 +1 位授课教师

20,202 人已注册

提供方

IBM 技能网络

课程信息

169,262 次近期查看

Bernard Marr defines Big Data as the digital trace that we are generating in this digital era. In this course, you will learn about the characteristics of Big Data and its application in Big Data Analytics. You will gain an understanding about the features, benefits, limitations, and applications of some of the Big Data processing tools. You’ll explore how Hadoop and Hive help leverage the benefits of Big Data while overcoming some of the challenges it poses. 
Hadoop is an open-source framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. Hive, a data warehouse software, provides an SQL-like interface to efficiently query and manipulate large data sets residing in various databases and file systems that integrate with Hadoop.

Apache Spark is an open-source processing engine that provides users new ways to store and make use of big data. It is an open-source processing engine built around speed, ease of use, and analytics. In this course, you will discover how to leverage Spark to deliver reliable insights. The course provides an overview of the platform, going into the different components that make up Apache Spark. 

In this course, you will also learn about Resilient Distributed Datasets, or RDDs, that enable parallel processing across the nodes of a Spark cluster.

可灵活调整截止日期

根据您的日程表重置截止日期。

可分享的证书

完成后获得证书

100% 在线

立即开始，按照自己的计划学习。

初级

7 weeks of study, 1-2 hours / week

完成时间大约为13 小时

英语（English）

对员工进行热门技能培训能否为您的公司带来益处？

体验 Coursera 企业版

您将学到的内容有

Explainthe impact of Big Data including use cases, tools, and processing methods.
ExplainApache Hadoop architecture, ecosystem, and practices, and userelatedapplications including HDFS, HBase, Spark, and MapReduce.
Apply Spark programming basics, including parallel programming basics forDataFrames, data sets, and Spark SQL.
UseSpark’s RDDsanddata sets, optimizingSparkSQLusing Catalyst and Tungsten, anduseSpark’s development and runtime environment options.

您将获得的技能

Apache Hadoop
SparkSQL
SparkML
Big Data
Apache Spark

可灵活调整截止日期

根据您的日程表重置截止日期。

可分享的证书

完成后获得证书

100% 在线

立即开始，按照自己的计划学习。

初级

7 weeks of study, 1-2 hours / week

完成时间大约为13 小时

英语（English）

对员工进行热门技能培训能否为您的公司带来益处？

体验 Coursera 企业版

授课教师

授课教师评分

4.41/5 （40 个评分）

Karthik Muthuraman

Software Engineer (Machine Learning)

IBM Center for Open-Source Data and AI Technologies

21,831 个学生

2 门课程

Aije Egwaikhide

Senior Data Scientist

IBM

406,925 个学生

6 门课程

提供方

IBM 技能网络

IBM is the global leader in business transformation through an open hybrid cloud platform and AI, serving clients in more than 170 countries around the world. Today 47 of the Fortune 50 Companies rely on the IBM Cloud to run their business, and IBM Watson enterprise AI is hard at work in more than 30,000 engagements. IBM is also one of the world’s most vital corporate research organizations, with 28 consecutive years of patent leadership. Above all, guided by principles for trust and transparency and support for a more inclusive society, IBM is committed to being a responsible technology innovator and a force for good in the world.

了解顶级公司的员工如何掌握热门技能

了解关于 Coursera for Business 的更多信息

授课大纲 - 您将从这门课程中学到什么

周

周 1

完成时间为 1 小时

What is Big Data?

完成时间为 1 小时

7 个视频（总计 42 分钟）, 2 个阅读材料, 2 个测验

7 个视频

What is Big Data?7分钟

Impact of Big Data5分钟

Parallel Processing, Scaling, and Data Parallelism7分钟

Big Data Tools and Ecosystem4分钟

Open Source and Big Data6分钟

Beyond the Hype4分钟

Big Data Use Cases5分钟

2 个阅读材料

Course Introduction2分钟

Summary & Highlights2分钟

2 个练习

Practice Quiz: Introduction to Big Data10分钟

Graded Quiz: Introduction to Big Data15分钟

周

周 2

完成时间为 2 小时

Introduction to the Hadoop Ecosystem

完成时间为 2 小时

6 个视频（总计 37 分钟）, 1 个阅读材料, 4 个测验

周

周 3

完成时间为 1 小时

Apache Spark

Build your skills when you turn your attention to the popular Apache Spark platform. Explore attribute and benefits of Apache Spark and distributed computing. You'll gain key insights about functional programming and Lambda functions. Explore Resilient Distributed Datasets (RDDs), Parallel Programming, resilience in Apache Spark and relate RDDs and Parallel Programming with Apache Spark. Dive into additional Apache Spark components and learn how Apache Spark scales with Big Data. Working with Big Data signals the need for working with queries, including structured queries using SQL. Learn about the functions, parts and benefits of Spark SQL and DataFrame queries, and discover how DataFrames work with SparkSQL.

完成时间为 1 小时

5 个视频（总计 24 分钟）, 1 个阅读材料, 3 个测验

周

周 4

完成时间为 2 小时

DataFrames and SparkSQL

Learn about Resilient Distributed Datasets (RDDs), their uses in Apache Spark, and RDD transformations and actions. You'll compare the use of datasets with Spark's latest data abstraction, DataFrames. You'll learn to identify and apply basic DataFrame operations. Explore Apache Spark SQL optimization. Learn how Spark SQL and memory optimization benefit from using Catalyst and Tungsten. Learn how to create a table view and apply data aggregation techniques. Fortify your skills guided via the hands-on lab.

完成时间为 2 小时

5 个视频（总计 26 分钟）, 1 个阅读材料, 4 个测验

周

周 5

完成时间为 4 小时

Development and Runtime Environment Options

Explore how Spark processes the requests that your application submits. Learn how you can track work using the Spark Application UI. Because Spark application work happens on the cluster, you need be able to identify Apache Cluster Managers, their components, benefits, and know how to connect with each cluster manager and how and when you might want to set up a local, standalone Spark instance. Next, learn about Apache Spark application submission, including use of Spark’s unified interface, ‘spark-submit’ and learn about options and dependencies. Developers now have the option of AIOps. Discover how to use Spark within AIOps and with Apache Spark application submission, including use of Spark’s unified interface, ‘spark-submit’, describe and apply options for submitting applications, identify external application dependency management techniques and list Spark Shell benefits. View and see recommended practices for Spark's static and dynamic configuration options. Round out your development knowledge with insights about Spark on Kubernetes. This module features hands-on Spark labs using IBM Cloud and Kubernetes.

完成时间为 4 小时

6 个视频（总计 33 分钟）, 2 个阅读材料, 7 个测验

6 个视频

Apache Spark Architecture5分钟

Overview of Apache Spark Cluster Modes6分钟

How to Run an Apache Spark Application6分钟

Using Apache Spark on IBM Cloud4分钟

Setting Apache Spark Configuration5分钟

Running Spark on Kubernetes 4分钟

2 个阅读材料

Summary & Highlights2分钟

4 个练习

Practice Quiz: Spark Architecture6分钟

Graded Quiz: Spark Architecture9分钟

Practice Quiz: Spark Runtime Environments6分钟

Graded Quiz: Spark Runtime Environments9分钟

周

周 6

完成时间为 3 小时

Monitoring & Tuning

Platforms and applications require monitoring and tuning to manage issues that inevitably happen. In this module you'll learn about connecting the Apache Spark user interface web server and using the same UI web server to manage application processes. Identify common Apache Spark application issues. Learn about debugging issues using the application UI and locating related log files. Discover and gain real-world knowledge about how Spark manages memory and processor resources via videos and in the available hands-on lab.

完成时间为 3 小时

5 个视频（总计 30 分钟）, 4 个阅读材料, 4 个测验

5 个视频

The Apache Spark User Interface5分钟

Monitoring Application Progress7分钟

Debugging Apache Spark Application Issues5分钟

Understanding Memory Resources5分钟

Understanding Processor Resources5分钟

4 个阅读材料

Summary & Highlights2分钟

Instructions for the Final Exam2分钟

Congrats & Next Steps2分钟

Team & Acknowledgements2分钟

3 个练习

Practice Quiz: Introduction to Monitoring & Tuning10分钟

Graded Quiz: Introduction to Monitoring & Tuning15分钟

Final Exam 1 小时 40 分

审阅

4.3

34 条评论

5 stars
62.64%
4 stars
20.11%
3 stars
10.34%
2 stars
2.87%
1 star
4.02%

来自INTRODUCTION TO BIG DATA WITH SPARK AND HADOOP的热门评论

由 RR 提供May 1, 2022

hands on lab and quizzes at the end of each session was very helpful

由 ND 提供Nov 8, 2022

All the thinks I need to know about Big Data, Spark, Hadoop and Hive and explained in details

由 JH 提供Dec 31, 2022

Muy util curso, sin embargo fue complicado es muy teórico

由 RS 提供May 7, 2022

Fantastic blend of theory and practical (labs). The labs are short and have concise material.

查看所有评论

常见问题

我什么时候能够访问课程视频和作业？
讲座和作业的访问权限取决于您的注册类型。如果您以旁听模式参加课程，则可以免费查看大多数课程资料。要访问评分作业并获得证书，您需要在旁听期间或之后购买证书体验。如果看不到旁听选项：
课程可能不提供旁听选项。您可以尝试免费试用，也可以申请助学金。
课程可能会改为提供'完整课程，没有证书'。通过此选项，您可以查看所有课程材料、提交所要求的作业，以及获得最终成绩。这也意味着您将无法购买证书体验。
我订阅此证书后会得到什么？
您注册课程后，将有权访问证书中的所有课程，并且会在完成作业后获得证书。您的电子证书将添加到您的成就页中，您可以通过该页打印证书或将其添加到您的领英档案中。如果您只想阅读和查看课程内容，可以免费旁听课程。

还有其他问题吗？请访问学生帮助中心。

培养员工技能，推动业务成果

探索 Coursera for Business

Introduction to Big Data with Spark and Hadoop

课程信息

您将学到的内容有

您将获得的技能