Suresh Payankannur

Showing posts with label ubuntu. Show all posts
Showing posts with label ubuntu. Show all posts

Tuesday, August 18, 2015

Ubuntu + Jenkins + Gradle + Embedded Tomcat + Port 80

Port 80 is a privileged port and normal users cannot listen on this port. I had a recent requirement to run some test cases on an embedded tomcat running on Port 80 during Jenkins build. Here is the steps worked for me to achieve this.

Gradle File

apply plugin: 'java'
apply plugin: 'war'

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath 'com.bmuschko:gradle-tomcat-plugin:2.2.2'
  }
}

apply plugin: 'com.bmuschko.tomcat'

tomcat {
  httpPort = 80
  contextPath = "/"
}

Setup authbind

authbind will allow non-root user to listen on privileged ports.
sudo apt-get install authbind
sudo touch /etc/authbind/byport/80
sudo chmod 500 /etc/authbind/byport/80 
sudo chown jenkins /etc/authbind/byport/80

Configure Jenkins

Modify the Jenkins run command (last line in do_start routine)to the following
$SU -l $JENKINS_USER --shell=/bin/bash -c "$DAEMON $DAEMON_ARGS -- authbind --deep $JAVA $JAVA_ARGS -jar $JENKINS_WAR $JENKINS_ARGS" || return 2

Friday, April 25, 2014

Using different credentials with ssh-agent

When using ssh directly, you can use the -i <identity-file> option to specify different key files. But when using programs like git which uses ssh under the covers, it is not possible to directly specify a different key file. In such situations, ssh-agent can be used to supply the required identity for any programs running within the context.
ssh-agent bash
ssh-add <private-key-file>
git push

Thursday, April 24, 2014

Remove packages marked as 'deinstall' in Ubuntu

To list left over configuration files and binaries for Ubuntu packages
dpkg --get-selections | grep -i deinstall
To completely remove packages including binaries and configuration files
dpkg --get-selections | grep -i deinstall | awk -F ' ' '{print $1}' | xargs sudo apt-get purge -y -

Thursday, May 24, 2012

Migrating from Gitosis to Gitolite on Ubuntu 12.04

Ubuntu 12.04 (Precise Pangolin) removed the support for Gitosis Git server. I have been using to host my projects locally. This forced me to look for alternatives. This link http://askubuntu.com/questions/126097/ubuntu-12-04-gitosis-no-longer-available discusses the alternatives.

I decided to go ahead and use Gitolite, which also provides migration from Gitosis. Here are the steps to migrate from Gitosis to Gitolite.

git-server: Where the git repository is hosted
git-client: A windows git client. I am using command line git from Cygwin
gitadmin: The user account under which the git repository is managed.
gituser: The windows user name under which the git client operations are performed.

Copy the existing public key to the server.
git-client> scp /home/gituser/.ssh/id_rsa.pub gitadmin@git-server:/tmp/  
Disable the gitosis and install gitolite
git-server> sudo apt-get install gitolite
git-server> su gitadmin
git-server> cd /home/gitadmin
git-server> mv .ssh/autorized_keys .ssh/autorized_keys.back
git-server> cp -r respositories repositories.gitosis
git-server> cd repsitories/gitosis-admin.git/hooks
git-server> mv post-update post-update.sample
git-server> cd /tmp
git-server> gl-setup id_rsa.pub
Make changes to the gitolite configurations
git-client> git clone git://github.com/sitaramc/gitolite
git-client> git clone gitadmin@git-server:gitolite-admin.git
git-client> gitolite/convert-gitosis-conf < gitosis-admin/gitosis.conf >> gitolite-admin/conf/gitolite.conf
git-client> cd gitolite-admin/conf
git-client> vi gitolite.conf
Modify the gitolite.conf and delete the gitosis-admin repo. Also modify any user names which do not have a dot '.' in them as discussed in the migration document here: http://sitaramc.github.com/gitolite/gsmigr.html. In my setup, the sample configuration will look like:



Copy the existing keys
git-client> gitosis-admin/keydir/*  gitolite-admin/keydir
Delete any duplicate keys as discussed here: http://sitaramc.github.com/gitolite/gsmigr.html

Commit the changes
git-client> cd gitolite-admin
git-client> git commit -am 'Moved to gitolite repository'
git-client> git push

Tuesday, May 1, 2012

Installing a Git Server on Ubuntu 11.10

Note: This will not work with Ubuntu 12.04, as the support for gitosis is removed in this release. See http://askubuntu.com/questions/126097/ubuntu-12-04-gitosis-no-longer-available for details. For migration from gitosis to gitolite, see this blog post: http://www.sureshpw.com/2012/05/migrating-from-gitosis-to-gitolite-on.html

There are a number of articles and blogs on how to use Git. But not very much information on how to setup your own Git server. Here are the steps on how to setup your Git server using gitosis on Ubuntu and use a Git client from Windows (cygwin in my case).

git-server: The server where the git repository is hosted.
git-client: The windows client accessing git
gituser: Windows user name using the git client
git: The username under which the git repository is created on the git-server.

First of all make sure that the Ubuntu server has the latest updates and install the required tools
git-server> sudo apt-get update
git-server> sudo apt-get dist-upgrade
git-server> sudo apt-get install python-setuptools
Install gitosis and add user named git on the git-server who will manage the git repository.
git-server> sudo apt-get install git-core gitosis
git-server> sudo adduser --system --shell /bin/sh --gecos 'git version control' \
                         --group --disabled-password --home /home/git git            
On the client machine, create a SSH public/private key for the user to access Git. Then provide permissions on the git server for this user.
git-client> ssh-keygen -t rsa
This will create private/public keys under $HOME/.ssh directory. On a cygwin platform, it will be under /home/gituser directory. Now upload the public key to git-server.
git-client> scp /home/gituser/.ssh/id_rsa.pub gitadmin@git-server:/tmp/
Initialize gitosis admin repository
git-server> sudo -H -u git gitosis-init < /tmp/id_rsa.pub
git-server> sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update
Add your new project to git by modifying the git-admin configuration.
git-client> git clone git@git-server:gitosis-admin.git
git-client> cd gitosis-admin
git-client> vi gitosis.conf
The gitosis.conf has one project, gitosis-admin. Duplicate the [gitosis-admin] section and modify for your new project. The results will be as given below:



Commit and push the new changes to gitosis.conf file.
git-client> git commit -a -m 'Added new project my-new-project'
git-client> push
Import files to the new project
git-client> cd my-new-project
git-client:~/my-new-project/> git init
git-client:~/my-new-project/> git add .
git-client:~/my-new-project/> git commit -m 'Initial import of files to my-new-project'
git-client:~/my-new-project/> git remote add origin git@git-server:my-new-project.git
git-client:~/my-new-project/> git push -u origin master
Now the project is available in the remote git repository. Any new user can get the code by simply cloning the repository
git clone git@git-server:my-new-project.git

References

  1. http://www.hackido.com/2010/01/installing-git-on-server-ubuntu-or.html
  2. http://blog.jasonmeridth.com/2010/05/22/gitosis-and-gitweb-part-1.html
  3. http://thelucid.com/2008/12/02/git-setting-up-a-remote-repository-and-doing-an-initial-push/
  4. https://help.ubuntu.com/community/Git

Monday, April 30, 2012

Static IP address under Ubuntu 11.10

Ubuntu 11.10 has changed the network startup process. The old way of configuring network settings in /etc/network/interfaces and /etc/resolv.conf do not work anymore. The new startup will rewrite the /etc/resolve.conf by erasing your name servers. Making no connectivity to outside world.

Looks like there are no other directives or configurations which instruct the new startup not to modify the /etc/resolv.conf. So the choice is to either use the new configuration or uninstall the Network Manager and use the old style configurations.

Using Network Manager Applet

Settings->Network->Your-Network-Connection->Configure->IPV4 Settings will bring up the following dialog. The static IP address and the rest of the configuration parameters can be specified here, including the gateway and DNS server.


Or manually modify the new Network Manager configuration file instead of using the GUI

The Network Manager saves the configuration in the file /etc/NetworkManager/system-connections directory. Open th the file using sudo permissions to edit it.



As you can see the [ipv4] section has the information provided in the GUI.

Uninstall Network Manager and use old style configuration

sudo apt-get remove network-manager network-manager-gnome
Change the /etc/resolv.conf to add the nameserver.
nameserver 192.168.72.2
Change the /etc/network/interfaces to add the static IP configuration
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.72.131
netmask 255.255.255.0
gateway 192.168.72.2
Then Restart the networking.

Saturday, September 3, 2011

Ubuntu + Redmine + Apache2 + Passenger + Enterprise Ruby + MySQL

This post describes how to install Redmine on Ubuntu, behind Apach2 and Passenger Mode using Enterprise Ruby. I am using Amazon 64-bit Ubuntu AMI.


Install MySQL

sudo apt-get install mysql-server
sudo apt-get install mysql-client
sudo apt-get install libmysqlclient-dev
mysql -u root -p
mysql> create database redmine set character utf8;
mysql> grant all privileges on redmine.* to '<redmine-user>'@'localhost' identified by '<redmine-pass>' with grant option;
mysql> flush privileges;

Install Apache

sudo apt-get install apache2

Install Redmine

cd /opt
sudo wget http://rubyforge.org/frs/download.php/75097/redmine-1.2.1.tar.gz
sudo tar xvzf redmine-1.2.1.tar.gz
sudo ln -s redmine-1.2.1 redmine
sudo chown -R root:root redmine
sudo chown -R root:root redmine-1.2.1
cd redmine
sudo chmod 0755 log
cd log
sudo touch production.log
sudo chmod 0666 production.log
cd ../config
sudo cp database.yml.example database.yml
// Edit the database.yml and change the settings on production to reflect the MySQL database name, db user name and db password created in the above section.

Install Enterprise Ruby

mkdir ~/tmp
cd ~/tmp
wget http://rubyenterpriseedition.googlecode.com/files/ruby-enterprise-1.8.7-2011.03.tar.gz
tar xvzf ruby-enterprise-1.8.7-2011.03.tar.gz
cd ruby-enterprise-1.8.7-2011.03
sudo ./installer
This step will tell you what packages are missing. Follow the instructions and install all the required packages. Then re-run the installer. The installer will compile the ruby enterprise edition and will ask the directory to install. By default, it will be installed in the /opt directory.


In my case, the compiler came back with the following error:

ossl_ssl.c:104:1 - 'SSLV2_method' undeclared here (not in a function)
ossl_ssl.c:105:1 - 'SSLV2_server_method' undeclared here (not in a function)
ossl_ssl.c:106:1 - 'SSLV2_client_method' undeclared here (not in a function)

The solution was to comment out these lines from the source code ~/tmp/ruby-enterprise-1.8.7-2011.03/source/ext/openssl/ossl_ssl.c and re-run the installer.


Wait for the installer to finish. It will take a while.

Install Passenger

cd /opt/ruby-enterprise-1.8.7-2011.03/bin
sudo ./passenger-install-apache2-module

This step will tell you missing packages and how to install them.

Install some of the missing gems

cd /opt/ruby-enterprise-1.8.7-2011.03/bin
sudo ./gem install --no-rdoc --no-ri rack -v=1.1.0
sudo ./gem install  --no-rdoc --no-ri i18n -v=0.4.2
sudo ./gem install --no-rdoc --no-ri gruff
sudo ./gem install --no-rodc --no-ri fastercsv

Configure Apache

  • Edit /etc/apache2/sites-available/default and add the following, assuming you are hosting the redmine at the root.
    DocumentRoot /opt/redmine/public
    PassengerMaxPoolSize 10
    PassengerMinInstances 3
    PassengerHighPerformance on
    PassengerRoot /opt/ruby-enterprise-1.8.7-2011.03/lib/ruby/gems/1.8/gems/passenger-3.0.8
    PassengerRuby /opt/ruby-enterprise-1.8.7-2011.03/bin/ruby
    <Directory /opt/redmine/>
        Options FollowSymLinks -MultiViews
        AllowOverride none
        Order allow, deny
        allow from all
        PassengerAppRoot /opt/redmine
    </Directory>
    
  • To load the passenger module, create a new file /etc/apache2/mods-available/passenger.load and add the following line:
    LoadModule passenger_module /opt/ruby-enterprise-1.8.7-2011.03/lib/ruby/gems/1.8/gems/passenger-3.0.8/ext/apache2/mod_passenger.so
    
  • Enable the module
    cd /etc/apache2/modes-enabled
    sudo ln -s ../modes-available/passenger.load passenger.load 
    

Start the apache


sudo /etc/init.d/apach2 start

Point the browser http://localhost and see if any libraries or gems are missing. Also inspect the /var/log/apache2/error.log and /opt/redmine/log/production.log for any issues.

Sunday, July 31, 2011

Confluence + Tomcat + Ubuntu + MySQL

To install the EAR/WAR version of Confluence follow these steps. This assumes basic knowledge on Ubuntu, MySQL, Tomcat etc.
  1. Download the latest EAR/WAR version
  2. Unpack the distribution
        % sudo tar xvf confluence-3.4.6.tar -C /opt
    
  3. Create symlink wiki for the confluence distribution
        % cd /opt
        % sudo ln -s confluence-3.4.6 wiki
        % sudo chown -R tomcat6:tomcat6 wiki
      
  4. Create the data directory
        % sudo mkdir /var/data/wiki
        % sudo chown -R tomcat6:tomcat6 /var/data/wiki
      
  5. Edit the /opt/wiki/confluence/WEB-INF/classes/confluence-init.properties and set the value of variable confluence.home to /opt/wiki/confluence
  6. Edit /etc/default/tomcat6 and adjust the JVM settings
    JAVA_OPTS="-Djava.awt.headless=true -Xmx640m -XX:+UseConcMarkSweepGC -XX:MaxPermSize=256m"
    
  7. Modify /opt/wiki/confluence/WEB-INF/classes/log4j.properties so that the log files will be produced under /var/log/tomcat6
    • Comment the line for ConfluenceHomeLogAppender
    • Uncomment the line for RollingFileAppender
    • Set the log location to /var/log/tomcat6/atlassian-confluence.log
    • Set the debug threshold to WARN
  8. Create a confluence database
        % mysqladmin -u root -p create wiki
        % mysql -u root -p
        % mysql> grant all privileges on wiki.* to 'wikiuser'@'localhost' identified by 'wikipass' with grant option;
        % mysql> commit;
        % mysql> flush all privileges;
      
  9. Setup the confluence context: In the /var/lib/tomcat6/conf/Catalina/localhost directory, create a file called confluence.xml with the following contents:
    <Context path="/wiki" docBase="/opt/wiki/confluence" debug="0" reloadable="0"/>
    
  10. Restart tomcat
        % sudo /etc/init.d/tomcat6 restart
    

  11. Point the browser to localhost:8080/wiki and follow the instructions

Install VMWareTools on Ubuntu Server

I have been running Ubuntu Server Guest OS on OSX. Ubuntu server is installed without any X support. So to install VMWare Tools, which will allow to share folders between the Guest OS and host, follow these instructions:

  1. Upgrade the server
        % sudo apt-get update
        % sudo apt-get dist-upgrade
      
  2. Install the required packages to compile VMWare Tools
        sudo apt-get install build-essential linux-headers`uname -r` psmic
      
  3. Reboot the server
        % sudo reboot now
      
  4. Go to the menu bar Virtual Machine -> CD/DVD -> Choose Disk Image. Choose the file /Library/Application Support/VMWare Fusion/isoimages/linux.so
  5. Mount the CDROM on the Ubuntu Server
         % sudo mount /dev/cdrom /media/cdrom
       
  6. Copy the VMWare Tools from CDROM to the local
        % cp /media/cdrom/VMWareTools-8.4.5-332101.tar.gz ~/tmp
      
  7. Unpack VMWare Tools
        % tar xvf VMWareTools-8.4.5-332101.tar.gz
      
  8. Install the VMWareTools. Provide default values for all the questions.
        % cd ~/tmp/vmware-tools-distrib
        % ./vmware-install.pl
      
  9. Reboot the server
        % sudo reboot now
      

Installing Apache + Tomcat on Ubuntu

  1. Install the packages
        sudo apt-get update
        sudo apt-get dist-upgrade
        sudo apt-get install apache2
        sudo apt-get install tomcat6
        sudo apt-get install libapache2-mod-jk
      
  2. Adjust the startup parameters like heap space, permgen etc. by modifying the file /etc/default/tomcat6
  3. Uncomment the AJP connector from the server configuration by modifying /var/lib/tomcat6/conf/server.xml
  4. Add the jk worker properties file by creating a new file /etc/apache2/jk_workers.properties with the following content:
        # List of workers
        worker.list = local, status
    
        # Local worker
        worker.local.type=ajp13
        worker.local.host=localhost
        worker.local.port=8009
    
        # Status worker
        worker.status.type=status
      
  5. Add a new file /etc/apache2/mods_available/jk.conf with the following content:
        JkWorkersFile /etc/apache2/jk_workers.properties
        JkShmFile     /var/log/apache2/mod_jk.shm
        JkLogFile     /var/log/apache2/mod_jk.log
        JkLogLevel    info
        JkLogStampFormat "[%a %b %H:%M:%S %Y]"
      
  6. Enable the mods
          % cd /etc/apache2/mods-enabled
          % sudo ln -s ../mods-available/jk.conf jk.conf
        
  7. Edit /etc/apache2/sites-available/default and add the JK Mount points. At the end of the file, in the same virtual host definition, add the following:
        JkMount /servlet* local
        JkMount /tomcat-status status
      
  8. Reboot the server
          % sudo reboot now
        

Install Java6 on Ubuntu

Java6 is not available from standard repositories. In order to install Java6, a new repository should be added. In addition, some new packages needs to be installed.

  1. Get additional packages
    sudo apt-get install python-software-properties
    
  2. Add new repository
    sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
    
  3. Update
    sudo apt-get update
    
  4. Install Java
    sudo apt-get install sun-java6-jdk
    
  5. Set JAVA_HOME
    export JAVA_HOME=/usr/lib/jvm/java-6-sun
    

Blog Archive

Scroll To Top