The HFactory Tools for HDInsight service automates the deployment of HFactory Tools on top of a HDInsight cluster. Just follow the below simple steps and be started in no time!
A resource group
You need a new resource group to put your HDInsight cluster into:
A virtual network
Create a virtual network in your resource group using the Resource Manager deployment method.
Then give the name of the network and its subnet:
A network security group
Create a network security group using the Resource Manager deployment method and specify its name.
Refresh the resource group to access it when deployed:
Add inbound rules to the network security group:
First HTTPS from everywhere or at least this four Azure ips to allow monitoring of the HDInsight cluster by Azure (22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206):
Then SSH to be able to connect to the VM and the head node of HDInsight:
Create the cluster
In your resource group create a new HDInsight cluster.
Select an available name:
Select the HBase type:
Give the credentials:
Define the storage account and container:
Define the cluster size:
*** Important: use your previously created network *** Select your Network and subnet:
Finally check that the resource group is the correct one and deploy the cluster.
HFactory Server VM
Get the quorum
Once the HDInsight cluster is deployed, go to the cluster dashboard using the address given in the azure portal:
Connect to it using the credentials given during the installation:
Go into the HBase configuration by clicking on HBase first then Configs:
Use the search box to get the quorum property and copy this value that you will need to configure the HFactory VM:
Create the VM
Select HFactory Tools for HDInsight from the Azure marketplace or use this direct link and click create:
Provide a name, a user, a password or a SSH-key, a public ip:
Most importantly use the same network and subnet than the HDInsight cluster:
If you have configured an SSH access for all, simply connect to the VM with SSH. If you only have access to the namenode add the following lines to your $HOME/.ssh/config files:
Host HFTForHDI User hfactory HostName 172.17.0.6 ProxyCommand ssh email@example.com nc %h %p 2> /dev/null IdentityFile ~/keys/id_rsa
You need to change the
User to the username of your VM,
Hostname to its private ip. You can drop the IDentityFile line if you are using your default key or the password method. Finally change
ssh firstname.lastname@example.org to the SSH connection string that you can find here:
Once SSH is properly configured connect to the VM and give the zookepper quorum when asked to.
You can contact Ubeeko email@example.com for a license to copy to
/usr/local/hfactory-server/conf/hfactory.license to upgrade from the developper license included in the VM.
You can now start the server with:
sudo systemctl start hfactory
You can also make the server start on boot or reboot with:
sudo systemctl enable hfactory
By default HFactory server listen on port 30100, you can change that by adding the
hserver.listen.port property to the
You can copy your applications into
/usr/local/hfactory-server/apps and start using them.