Cdh media wizard

Hi CDH, Cloudera Manager, and Cloudera Navigator users,We’re pleased to announce the release of Cloudera Enterprise 5.4 (comprising CDH 5.4, Cloudera Manager 5.4, and Cloudera Navigator 2.3).Cloudera Enterprise 5.4 (Release Notes) reflects critical investments in a production-ready customer experience through governance, security, performance and deployment flexibility in cloud environments. It also includes support for a significant number of updated open standard components–including Apache Spark 1.3, Impala 2.2, and Apache HBase 1.0 (as well as unsupported beta releases of Hive-on-Spark data processing and OpenStack deployments).Recently Cloudera made the upgrade process considerably easier via an improved CDH upgrade wizard; see details about that wizard here and best practices here. (Note: Due to metadata format changes in Apache Hadoop 2.6, upgrading to CDH 5.4 and later from any earlier release requires an HDFS metadata upgrade, as well. Note also that for parcel-based installs of CDH 5.4, Cloudera Manager 5.4 is required.)SecuritySSL and Kerberos support in Apache Flume for the Thrift source and sink.SSL support across Cloudera Search (Solr and all integrations with CDH).Cluster-wide redaction of sensitive data in logs is now possible.HBase impersonation in Hue allows your client to authenticate to HBase as any user, and to re-authenticate at any time.Solr metadata stored in ZooKeeper can now be protected by ZooKeeper ACLs.Kerberos is now supported for Apache Sqoop2.PerformanceIncludes beta release of Hive-on-Spark as an option for improved Hive data processing performance (unsupported).MultiWAL support for HBase RegionServers allows you to increase throughput when a region writes to the write-ahead log (WAL).You can now store medium-sized objects (MOBs) up to 10MB in size directly in HBase while maintaining read and write performance.A new Kafka connector for Spark Streaming avoids the need for the HDFS WAL.Hue pages render much faster.Data Management and GovernanceExpanded coverage in Cloudera NavigatorImpala (CDH 5.4 and higher) lineageCloudera Search (CDH 5.4 and higher) auditingAuditing of Navigator activity, such as audit views, metadata searches, and policy editingAvro and Parquet schema inferencePlatform enhancementsRedesigned metadata search provides autocomplete, faster filtering, and saved searchesSAML for single sign-onCloud DeploymentsOpenStack deployments are now possible as an unsupported beta.HBase support on Microsoft Azure.Real-Time ArchitectureCloudera Distribution of Apache Kafka 1.3 installs by default and is supported for production use.Spark Streaming now has a receiver-less “direct” connector for Kafka.New or Updated Open Source ComponentsApache Hadoop 2.6Apache HBase 1.0Apache Hive 1.1Apache Oozie 4.1Apache Solr 4.10.3Apache Spark 1.3Cloudera Distribution of Apache Kafka 1.3Hue 3.7Impala 2.2Kite SDK 1.0New/Updated OS & Java SupportRHEL 6.6/CentOS 6.6/OEL 6.6 (UEK3)JDK8u40, JDK7u75Over the next few weeks, we’ll publish blog posts that cover some of these features in detail. In the meantime:Download Cloudera Enterprise 5.4Explore documentationAs always, we value your feedback; please provide any comments and suggestions through our community forums. You can also file bugs via issues.cloudera.org.

It is affectionately said that what Oracle is to Relational Database, Cloudera is to Hadoop. Most of the Hadoop aspirants, at the beginning of their Hadoop development learning curve, fiddle with the setting up of CDH, some able to do it smoothly (Cloudera has put up an incredibly exhaustive installation guide), some requires to really sweat it out (few finer details and prerequisites are either missing or not enough emphasized upon) and very few actually gives it up losing their way in the verbose and cover-all-cases installation guide(multiple way of set up and different set of instructions for different linux OS; sometimes too much of variations for impatient starters). So there is a target audience for one more set up document which is leaner, meaner and streamlined with only one (the most preferred) variation. Try it at home!!CDH 5.x.x RequirementsOperating System : Ubuntu :Precise (12.04) - Long-Term Support (LTS): 64-bitCDH 5.3.x runs on both Trusty (14.04) and Precise (12.04)Database :MySQL server version 5.5JDK :Oracle JDK 1.7.0_67Internet Protocol& Access :Protocol: IPv4Internet access to allow the wizard to install software packages or parcels from archive.cloudera.comWays of InstallationThere are two ways of installations1. Automated method using Cloudera ManagerCloudera Manager automates the installation and configuration of CDH 5.Note: Installing user to have root or password-less sudo SSH access to cluster's machines.This is our preferred installation path and rest of the document cater to this need only2. Manual methods:Download the CDH 5 1-click Install package OR Add the CDH 5 repository OR Build your own

Installation and configuration. There is one UI for each of the blow tasks. The wizard lets you:Select the version of Cloudera Manager to install.Find the cluster hosts you specify via hostname and IP address ranges.To enable Cloudera Manager to automatically discover hosts on which to install CDH and managed services, need to enter the cluster hostnames or IP addresses. You can also specify hostname and IP address ranges. For example:10.1.1.[1-4] --> 10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4host[1-3].company.com --> host1.company.com, host2.company.com, host3.company.comConnect to each host with SSH to install the Cloudera Manager Agent and other componentsOptionally installs the Oracle JDK on the cluster hosts if not pre-installedInstall CDH and managed service packages or parcels.Need to select the repository type to use for the installation: parcels or packages.Repository Type: parcelsBenefit: parcels provide a mechanism for upgrading the packages installed on a cluster from within the Cloudera Manager Admin Console with minimal disruption.Above figures are indicative. For example Cloudera search is included in CDH 5.x, SOLR might be included in CDH distribution itself.Configure CDH and managed services automatically and start the services.Uninstall ClouderaIf you have come to this procedure because your installation did not complete successfully, and you want to proceed with the installation, do the following:Remove files and directories$ sudo rm -Rf /usr/share/cmf /var/lib/cloudera* /var/cache/yum/cloudera*Run the installer againCaution: If you need to rerun the installer file (.bin) it may get locked. UI of installer will then wait forever to get the required file system locks before actual installation. You need to delete the lock