First steps - How to start your first freetz package

Foreword and Motivation

In order to create your first freetz package, you may want to find a project which is both interesting enough to be integrated into freetz to work on your box and also easy enough to be compiled.

A good example for your first steps might be the HTTP tunnel server. With httptunnel you can tunnel TCP connections over the http protocol and thus get access to your box even through restrictive proxies, e.g. such proxies where only ports 80 (http) and 443 (https) are open. More details can be read here.

Where to Ask Questions and to Discuss

This thread in the IPPF is the right place where you can read about the evolution of this project, with a lot of really helpful hints from the gurus.
This thread is also good for asking questions and for discussion.

Build Environment

There are different platforms which you may use to build your package, but usually all involve Linux in some way. The following steps were performed in this environment:

• AVM Fritzbox 7170 (Germany) with freetz-1.0
• StinkyLinux 1.06 in a VM on a Macbook

Very helpful information on make-targets such as menuconfig, toolchain, precompiled, recover etc. may be retrieved in the HowTos section of the Freetz wiki.

Toolchain

The toolchain is built automatically with "make" (see below).

If you want to have your toolchain ready at an earlier stage, you can create it now:
Go to your build environment and change to your freetz directory, usually ~/freetz-trunk.

Then create your toolchain:

$ make toolchain

FINISHED: toolchain/kernel/ - glibc compiler for the kernel
          toolchain/target/ - uClibc compiler for the userspace

In order to build your new package manually (without integration in the freetz build system), you should see that your toolchain is also included in the search path.

If you do not want to build it manually, but rather within the freetz build system, the following commands are not necessary:

export CC="mipsel-linux-gcc"
./configure --build=i386-linux-gnu --target=mipsel-linux --host=mipsel-linux
make
mipsel-linux-strip hts htc

Use of the "empty" Package as Starting Point

Go to ~/freetz-trunk/make/empty. This package will serve as your starting point to build your own httptunnel package.
There are three files: Config.in, empty.mk, Makefile.in (and a directory ".svn").

For your http tunnel project you may copy the complete "empty" directory with the three files as a new package. Let us call it "httptunnel".
Please go into that new "httptunnel" directory and remove the sub-directory ".svn". You will not need it. Now it should look like this:

-rw-r--r--   1 slightly slightly  480 2008-06-07 08:17 Config.in
-rw-r--r--   1 slightly slightly  701 2008-06-07 08:17 empty.mk
-rw-r--r--   1 slightly slightly   64 2008-06-07 08:17 Makefile.in

Rename "empty.mk" to "httptunnel.mk", because this is what your project is about now: httptunnel.

Now let us have a look at the "Config.in" file. Open it with your favorite editor, and it should look like this:

config FREETZ_PACKAGE_EMPTY
        bool "Empty 0.6.15b"
        select FREETZ_LIB_libutil
        default n
        help
                empty is an utility that provides an interface to execute and/or
                interact with processes under pseudo-terminal sessions (PTYs).
                This tool is definitely useful in programming of shell scripts
                designed to communicate with interactive programs like telnet,
                ssh, ftp, etc. In some cases, empty can be the simplest
                replacement for TCL/expect or other similar programming tools.

In this file you basically find the package name (bool) and a short help text.

You should change this to reflect your http tunnel project. Please note that the line "select FREETZ_LIB_libutil" is not necessary for your project, thus remove it:

config FREETZ_PACKAGE_HTTPTUNNEL
        bool "httptunnel 3.0.5 (binary only)"
        default n
        help
                httptunnel is a utility that provides a HTTP tunnel server on your box.

The next file "httptunnel.mk" (copied from "empty.mk") should be edited like this:

$(call PKG_INIT_BIN, 3.0.5)
$(PKG)_SOURCE:=$(pkg)-$($(PKG)_VERSION).tar.gz
$(PKG)_SITE:=http://www.nocrew.org/software/httptunnel
$(PKG)_BINARY:=$($(PKG)_DIR)/hts
$(PKG)_TARGET_BINARY:=$($(PKG)_DEST_DIR)/usr/bin/hts


$(PKG_SOURCE_DOWNLOAD)
$(PKG_UNPACKED)
$(PKG_CONFIGURED_CONFIGURE)

$($(PKG)_BINARY): $($(PKG)_DIR)/.configured
        PATH="$(TARGET_PATH)" \
                $(MAKE) -C $(HTTPTUNNEL_DIR) \
                CC="$(TARGET_CC)" \
                CFLAGS="$(TARGET_CFLAGS)"

$($(PKG)_TARGET_BINARY): $($(PKG)_BINARY)
        $(INSTALL_BINARY_STRIP)

$(pkg):

$(pkg)-precompiled: $($(PKG)_TARGET_BINARY)

$(pkg)-clean:
        -$(MAKE) -C $(HTTPTUNNEL_DIR) clean

$(pkg)-uninstall:
        $(RM) $(HTTPTUNNEL_TARGET_BINARY)

$(PKG_FINISH)

Explanation:

$(call PKG_INIT_BIN, 3.0.5)

This defines the version of your package.

$(PKG)_SOURCE:=httptunnel-$($(PKG)_VERSION).tar.gz

This defines the file name of the package source code, which has to be exactly as the filename on the server where the file is located.

$(PKG)_SITE:=http://www.nocrew.org/software/httptunnel

This defines the basis of the download URL from where the package source code will be downloaded during the build process.
In this case, the complete path would be: http://www.nocrew.org/software/httptunnel/httptunnel-3.0.5.tar.gz

$(PKG)_BINARY:=$($(PKG)_DIR)/hts

This defines the file name of the binary in the source directory.

$(PKG)_TARGET_BINARY:=$($(PKG)_DEST_DIR)/usr/bin/hts

This defines the path of the binary of your package where it will be stored on your box when your new package and FW is installed on your box.

Add New Package for "make menuconfig"

In order to see your new package when calling "make menuconfig" you need to add a line to ~/freetz-trunk/make/Config.in like this:

menu "Testing"
...
source make/httptunnel/Config.in
...

In this case, we add the new package "httptunnel" in the "testing" section. Of course you could add it elsewhere, but the other areas are reserved for already tested packages in production. Well, if you already knew that your new package was rather unstable, you could add it under "unstable". Anyway, the syntax is always the same as laid out above.

Now we also need to make sure that the following is changed, because "configure" needs to be called for this package to be built:

Go back to file "httptunnel.mk" and edit the line PKG_CONFIGURED_NOP to PKG_CONFIGURED_CONFIGURE:

$(PKG_SOURCE_DOWNLOAD)
$(PKG_UNPACKED)
$(PKG_CONFIGURED_CONFIGURE)

If you left it as PKG_CONFIGURED_NOP, this would mean that "configure" was not necessary to be called. However, with this package you will need it to be built.

Call Procedures "make menuconfig" and "make"

Now it is time to call "make menuconfig" and to choose your new package from the "packages/testing" section, where it should be available now for selection.

After you have finished and saved your various selections (starting just with your new package to test the build procedure), you will issue the "make" command.

Please be patient during the build procedure. Depending on your CPU(s) this may take some (longer) time.

A successful FW build with your new package included should end with these lines:

STEP 3: PACK
  Checking for left over Subversion directories
squashfs blocksize
  root filesystem: 65536
packing var.tar
creating filesystem image
merging kernel image
  kernel image size: 7354112 (max: 7798784, free: 444672)
packing 7170_04.57-freetz-1.0-2315M.de_20080611-222651.image
done.

FINISHED

Testing

Well, since testing depends on which package you have created, there is not much to say here - except that testing is easier if you did not include too many other packages, because these might interfere with your new package. Add more packages step by step only when you are pretty sure that it works.

Preparing New Package for Public Integration to Freetz Trunk

In order to create a file which displays the changes which would be needed in freetz to add your package, issue the following commands:

svn add make/httptunnel
svn diff ./make > patchfile

In our case "patchfile" may be called "httptunnel". Please note that there is no need for an extension here. You may only need an extension (e.g. .txt) for uploading it in the IPPF, because else it would not be recognized as a valid file for upload.

In addition you could even create a ready (and compressed) package of the (three) files which you had edited above:

tar cfz httptunnel.tar.gz make/httptunnel --exclude .svn

Well, that is it for the moment. I will add further stuff as I see fit. Of course everybody is invited to correct mistaked, add more information etc.
In case of questions, please do not hesitate to visit this thread in the IPPF. Thank you.