Archive for March, 2013

Howto resume SCP transfer

From time to time you will end up with an incomplete scp transfer. That gets annoying when the total transfer was high. The solution is to use rsync for resuming like so:

rsync -partial -progress --rsh=ssh user@host:/path/to/remote/file /path/local/file

# short version
rsync -P --rsh=ssh user@host:/path/to/remote/file /path/local/file

You can also add an alias so you dont have to remember all those flags. Add to your .bashrc or .bash_aliases

alias scpresume="rsync -P --rsh=ssh"

Git-Flow | How it’s used and why you should

What is Git-Flow about?

Git-Flow is a workflow for using Git in a way that makes continuous software development and lifecycle much better. It was first proposed by Vincent Driessen in early 2010. He then released some scripts that integrate into the git command. However many people / companies still havent heard of it.

It incorporates the typical software lifecycle steps: feature development, releasing a version, hotfixing.
Internally, its “just” a branching model, so it works with every git repo be it only local or with the big remote ones like Github, Gitorious.

Git-Flow concept

At first Git-Flow might be a bit confusing, but once you get the hang of it you won’t want to develop without it anymore. Have a look at this image while you are reading the explanation beneath and all should come clear.

Git branching model

Git branching model

Read more

iptables settings for outgoing FTP

Getting FTP to fully work with iptables can be a pain in the ass. Thinking of active and passive mode here. Even if you are familiar with iptables, its easier to copy/paste this rather than writing this down out of your head. (I am here refering to outgoing FTP connections, meaning you are acting as the client). So here are the rules you were looking for:

-A INPUT -p tcp --sport 21 -m state --state ESTABLISHED -j ACCEPT
-A INPUT -p tcp --sport 20 -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p tcp --sport 1024: --dport 1024: -m state --state ESTABLISHED -j ACCEPT

-A OUTPUT -p tcp --dport 21 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --dport 20 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --sport 1024: --dport 1024: -m state --state ESTABLISHED,RELATED,NEW -j ACCEPT

Basically what this does is tell iptables to open up FTP command port 21 and data port 20 for connection related to ones established on 21. It also allows the random ports >=1024 for related connections.
These rules apply for both active and passive connections.

Koji RPM Build System Installation Part 3

If you followed my previous articles you should by now have a rudimentary Koji system running.

Lets now proceed and add more components.

Koji-Web

As the name suggests, Koji-Web is a webinterface to Koji. It lets you view all your builds, packages, rpms, tasks and other useful info. However you cannot control everything about Koji with it. Its nonetheless good to have. So lets go:

yum install koji-web mod_ssl

Read more

Koji RPM Build System Installation Part 2

So in Part1 we started with setting up the SSL certificates. Now we are going deeper

Database Setup

Start with installing postgresql and setting up Koji users and schema.

yum install postgresql-server koji
service postgresql initdb
service postgresql start
useradd koji;passwd -d koji
su postgres;createuser koji;createdb -O koji koji
su koji; psql koji koji < /usr/share/doc/koji*/docs/schema.sql

Read more

Koji RPM Build System Installation Part 1

Introduction

So you decided to also take a shot at Koji, congrats. You won’t regret it.

At first, its helpful to understand the inner architecture of Koji for knowing when to look in which config files:

koji architecture

Koji architecture

Read more

Installing a RPM-based build system

When your Linux IT infrastructure has grown to a mature complex state and you heavily use continuous integration and automation, its time to think about how to deploy code efficiently.

You may already have Jenkins to build your own code and maybe even package it and installing with Puppet. But there is enough open-source software that you need to ./configure and compile yourself. Even if you then rpmbuild it yourself, its still not satisfying to install those RPM manually, especially if you already maintain your own repositories with Spacewalk.

A good solution is to build and package all your code into RPMs and import them into your very own repository which is available to all your system.
Read more