Request a callback

    Thank You!


    We will contact you shortly.
    +91 (120) 4119326
    info@datopic.com
    Schedule a Meeting

    Blogs

    Categories

    Share Links

    img

    VERTICA

    |  Oct 17,2018

    Vertica offers a software-based analytics platform designed to help organizations of all sizes monetize data in real time and at massive scale. Vertica is especially useful for businesses who want to keep track of their analytics and performance.

    Vertica Zest

    The Vertica Analytic Database consists of the following key features:

    1. Columnar Storage and Execution – column stores offer significant gains in performance, I/O, storage footprint, and efficiency when it comes to analytic workloads. With columnar storage the query only reads the columns needed to answer the query.
    2. Real-time loading and querying – with high query concurrency and the ability to simultaneously load new data into the system, Vertica can load data up to 10X faster than traditional row-store databases.
    3. Advanced Database Analytics – a set of Advanced In-Database Analytics allows you to conduct the analytics computations closer to the data. This provides immediate results from a single place without having to extract data from a separate environment.
    4. Database Designer and Administration Tools – these features allow you to tune and control Vertica with minimal administration effort.
    5. Advanced Compression – aggressive encoding and compression allows Vertica to dramatically improve analytic performance by reducing CPU, memory, and disk I/O at processing time. Vertica can reduce the original data size to up to 1/10th its original size.

    Massively Parallel Processing – a robust and scalable parallel processing solution provides active redundancy, automatic replication, failover, and recovery.

    Before You Install Vertica
    • Platform Requirements and Recommendations
    • Prepare Disk Storage Locations
    • Configuring the Network

    Platform Requirements and Recommendations
    You must verify that your servers meet the platform requirements described in Supported Platforms.

    BASH Shell

    All shell scripts included in Vertica must run under the BASH shell. If you are on a Debian system, then the default shell can be DASH. DASH is not supported. Change the shell for root and for the dbadmin user to BASH with the chsh command.
    For example:

    # getent passwd | grep root
    root:x:0:0:root:/root:/bin/dash
    
    # chsh
    Changing shell for root.
    
    New shell [/bin/dash]: /bin/bash
    Shell changed.
    

    Then, as root, change the symbolic link for /bin/sh from /bin/dash to /bin/bash:

    # rm /bin/sh
    # ln -s /bin/bash /bin/sh
    

    Log out and back in for the change to take effect.

    • Data Storage Recommendations
      All internal drives connect to a single RAID controller.
      The RAID array should form one hardware RAID device as a contiguous /data volume.
    • Validation Utilities
      Vertica provides several validation utilities that validate the performance on prospective hosts. The utilities are installed when you install the Vertica RPM, but you can use them before you run the install_vertica script.

    Prepare Disk Storage Locations
    We must create and specify directories in which to store your catalog and data files (physical schema). One can specify these locations while installing or configuring the database, or later during database operations. Both the catalog and data directories must be owned by the database administrator.

    The directory specified for database catalog files (the catalog path) is used across all nodes in the cluster. For example, if we specify /home/catalog as the catalog directory, Vertica uses that catalog path on all nodes. The catalog directory should always be separate from any data file directories.

    The data path you designate is also used across all nodes in the cluster. Specifying that data should be stored in /home/data, Vertica uses this path on all database nodes.

    Do not use a single directory to contain both catalog and data files. You can store the catalog and data directories on different drives, which can be either on drives local to the host (recommended for the catalog directory) or on a shared storage location, such as an external disk enclosure or a SAN.

    Before specifying a catalog or data path, be sure the parent directory exists on all nodes of the database. Creating a database in admin tools also creates the catalog and data directories, but the parent directory must exist on each node.

    There is no need to specify a disk storage location during installation. However, we can do so by using the –data-dir parameter to the install_vertica script.

    You can specify the disk storage location:

    • While Installing Vertica

    While installing Vertica, the –data-dir parameter in the install_vertica script lets you specify a directory to contain database data and catalog files. The script defaults to the database administrator’s default home directory /home/dbadmin.

    • While creating a database using the Administration Tools

    When we invoke the Create Database command in the Administration Tools, a dialog box allows you to specify the catalog and data locations. These locations must exist on each host in the cluster and must be owned by the database administrator.When we click OK, Vertica automatically creates the following subdirectories:

    catalog-pathname/database-name/node-name_catalog/data-pathname/database-name/node-name_data/

    For example, if we use the default value (the database administrator’s home directory) of /home/dbadmin for the Stock Exchange example database, the catalog and data directories are created on each node in the cluster as follows:

    /home/dbadmin/Stock_Schema/stock_schema_node1_host01_catalog/home/dbadmin/Stock_Schema/stock_schema_node1_host01_data
    • While Installing and configuring Management Console (MC)

    We can install MC before or after you install Vertica; however, consider installing Vertica and creating a database before you install MC.

    Configuring the Network
    This group of steps involve configuring the network. These steps differ depending on your installation scenario. A single node installation requires little network configuration, since the single instance of the Vertica server does not need to communication with other nodes in a cluster. For cluster and cloud install scenarios, you must make several decisions regarding your configuration.

    Vertica supports server configuration with multiple network interfaces. For example, you might want to use one as a private network interface for internal communication among cluster hosts (the ones supplied via the –hosts option to install_vertica) and a separate one for client connections.

    Optionally Run Spread on Separate Control Network

    If your query workloads are network intensive, you can use the –control-network parameter with the install_vertica to allow spread communications to be configured on a subnet that is different from other Vertica data communications.

    The –control-network parameter accepts either the default value or a broadcast network IP address (for example, 192.168.10.255 ).

    Configure SSH

      • Verify that root can use Secure Shell (SSH) to log in (ssh) to all hosts that are included in the cluster. SSH (SSH client) is a program for logging into a remote machine and for running commands on a remote machine.
      • If you do not already have SSH installed on all hosts, log in as root on each host and install it before installing Vertica. You can download a free version of the SSH connectivity tools from OpenSSH.
      • Make sure that /dev/pts is mounted. Installing Vertica on a host that is missing the mount point /dev/pts could result in the following error when you create a database:
         TIMEOUT ERROR: Could not login with SSH. Here is what SSH said:Last login: Sat Dec 15 18:05:35 2007 from v_vmart_node0001
      • Allow Passwordless SSH Access for the Dbadmin User
        The dbadmin user must be authorized for passwordless ssh. In typical installs, you won’t need to change anything; however, if you set up your system to disallow passwordless login, you’ll need to enable it for the dbadmin user.
    Install Vertica Using the Command Line

    Download and Install the Vertica Server Package

    1. Use a Web browser to log in to myVertica portal.
    2. Click the Download tab and download the Vertica server package to the Administration Host.
      Be sure the package you download matches the operating system and the machine architecture on which you intend to install it. In the event of a node failure, you can use any other node to run the Administration Tools later.
    3. If you are using sudo, skip to the next step. If you are root, log in to the Administration Host as root (or log in as another user and switch to root).
    $ su - root
    password: root-password
    #
    1. Use one of the following commands to run the RPM package installer:
    • If you are root and installing an RPM:
    # rpm -Uvh pathname
    • If you are using sudo and installing an RPM:
    $ sudo rpm -Uvh pathname
    • If you are using Debian:
    $ sudo dpkg -i pathname
    1. where pathnameis the Vertica package file you downloaded.
    Installing Vertica with the Installation Script

    Run the install script after you have installed the Vertica package. The install script is run on a single node, using a Bash shell, and it copies the Vertica package to all other hosts (identified by the –hosts argument) in your planned cluster.

    The install script runs several tests on each of the target hosts to verify that the hosts meet the system and performance requirements for a Vertica node. The install script modifies some operating system configuration settings to meet these requirements. Other settings cannot be modified by the install script and must be manually reconfigured.

    Basic Installation of Vertica

    As root (or sudo) run the install script. The script must be run by a BASH shell as root or as a user with sudo privileges. You can configure many options when running the install script.
    As root (or sudo) run the install script. The  script must be run by a BASH shell as root or as a user with sudo privileges. You can configure many options when running the install script.

    To perform a basic installation:
    As root:

    # /opt/vertica/sbin/install_vertica --hosts host_list --rpm package_name --dba-user dba_username
    

    Using sudo:

    $ sudo /opt/vertica/sbin/install_vertica --hosts host_list --rpm package_name --dba-user dba_username

    Important: If you place vertica_installation somewhere other than /opt/vertica, you need to create a symlink from that location to /opt/vertica. You need to create this symlink on all nodes in the cluster, otherwise the database will not start.

    Installing and Configuring Management Console (MC)

    This section describes how to install, configure, and upgrade Management Console (MC).
    You can install MC before or after you install Vertica; however, consider installing Vertica and creating a database before you install MC.

    Before You Install MC
    Management Console(MC) 0.x is compatible with Vertica server versions 6.1 and above.

    Driver Requirements for Linux SuSe Distributions

    The MC (vertica-console) package contains the Oracle Implementation of Java 7 JRE and requires that you install the unixODBC driver manager on SuSe Linux platforms. unixODBC provides needed libraries libodbc and lidodbcinst.

    Port Requirements
    When we use MC to create a Vertica cluster, the Create Cluster Wizard uses SSH on its default port (22).

    Port 5444 is the default agent port and must be available for MC-to-node and node-to-node communications.

    Port 5450 is the default MC port and must be available for node-to-MC communications.

    Firewall Considerations
    Make sure that a firewall or iptables are not blocking communications between the cluster’s database, Management Console, and MC’s agents on each cluster node.

    IP Address Requirements
    If you install MC on a server outside the Vertica cluster it will be monitoring, that server must be accessible to at least the public network interfaces on the cluster.

    Hardware Requirements

    Requirements CPU RAM Disk Space
    Minimum 4-core 4G 2G
    Recommended 8-core 8G 2G

    You can install MC on any node in the cluster, or its own dedicated node. When running the MC on a node in the cluster, note that MC shares RAM and time on CPU cores with other Vertica processes.

    Time Synchronization and MC’s Self-Signed Certificate
    When you connect to MC through a client browser, Vertica assigns each HTTPS request a self-signed certificate, which includes a timestamp. To increase security and protect against password replay attacks, the timestamp is valid for several seconds only, after which it expires.

    To avoid being blocked out of MC, synchronize time on the hosts in your Vertica cluster, and on the MC host if it resides on a dedicated server. To recover from loss or lack of synchronization, resync system time and the Network Time Protocol.

    SSL Requirements
    The openssl package must be installed on your Linux environment so SSL can be set up during the MC configuration process.

    File Permission Requirements
    On your local workstation, you must have at least read/write privileges on any files you plan to upload to MC through the Cluster Installation Wizard. These files include the Vertica server package, the license key (if needed), the private key file, and an optional CSV file of IP addresses.

    Monitor Resolution
    Management Console requires a minimum resolution of 1024 x 768, but Vertica recommends higher resolutions for optimal viewing.

    Installing Management Console
    We can install Management Console on any node planned to be included in the Vertica database cluster, as well as on its own, dedicated server outside the cluster.

    Install Management Console on the MC Server

    1. Download the MC package from the myVertica portal:
      vertica-console-current-version.Linux-distro)Save the package to a location on the target server, such as /tmp.
    2. On the target server, log in as root or a user with sudo privileges.
    3. Change to the directory where you saved the MC package.
    4. Install MC using your local Linux distribution package management system—rpm, yum, zypper, apt, dpkg. For example:

    Red Hat 6

    # rpm -Uvh vertica-console-current-version.x86_64.RHEL6.rpm

    Debian and Ubuntu

    # dpkg -i vertica-console-current-version.deb

    5. If you stopped the database before upgrading MC, restart the database.
    As the root user, use the following command:

    /etc/init.d/verticad start

    For versions of Red Hat 7/CentOS 7 and above, run:

    # systemctl start vertica-consoled

    6. Open a browser and enter the URL of the MC installation, one of the following:6. Open a browser and enter the URL of the MC installation, one of the following:
    IP address:
    https://ip-address:mc-port/
    Server host name:
    https://hostname:mc-port/

    7. By default, mc-port is 5450.

    8. If MC was not previously configured, the Configuration Wizard dialog box appears. Configuration steps are described in Configuring MC Section.
    If MC was previous configured, Vertica prompts you to accept the end-user license agreement (EULA) when you first log in to MC after the upgrade.

    Configuring MC

    After you install MC, you need to configure it through a client browser connection. An MC configuration wizard walks you through creating the Linux MC super administrator account, storage locations, and other settings that MC needs to run. Information you provide during the configuration process is stored in the /opt/vconsole/config/console.properties file.

    If you need to change settings after the configuration wizard ends, such as port assignments, you can do so later through Home > MC Settings page.

    Steps to Configure MC

    1. Open a browser session.
    2. Enter the IP address or host name of the server on which you installed MC (or any cluster node’s IP/host name if you already installed Vertica), and include the default MC port 5450. For example, you’ll enter one of:
      https://xx.xx.xx.xx:5450/ https://hostname:5450/
    3. Follow the configuration wizard.

    Visit us at datopic, share your business ideas or connect to join datopic ride on data science and cloud and take deep dive into big data solutions.

    Visit us at datopic, share your business ideas or connect to join datopic ride on data science and cloud and take deep dive into big data solutions.

    Share Links

    Your Comment:

    Request a Callback

    May I Help You ?    X