Checkout the repo
I am working with Fedora, so I use yum. I followed the build and configure instructions from here. In a nutshell, here are the steps to cloning the comm-central (Thunderbird) repo:
1) yum install mercurial - to get Mercurial, a version control tool.
2) Create a file called ".hgrc" in your home directory and put these in it:
[ui]
username=Firstname Lastname
merge=internal:merge
[diff]
git=1
showfunc=1
unified=8
[defaults]
commit=-v
Note: Obviously, replace Firstname, Lastname, and my.email... with your information.
3) Make sure you're in the directory that you want to put the repo in (for example, your home dir) then checkout the repo:
hg clone http://hg.mozilla.org/comm-central/ src-thunderbird
So now you should have a "src-thunderbird" folder in your current directory.
Building Thunderbird
1) Go to your src-thunderbird directory (cd src-thunderbird).
2) Create a .mozconfig file. (I use vi, so vi .mozconfig). This file will contain your build configurations for Thunderbird. The contents of mine are:
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-tbdebug
ac_add_options --enable-application=mail
ac_add_options --enable-debug
ac_add_options --disable-optimize
"obj-tbdebug" is the directory where all my object files will end up (you can change what you call yours). I will refer to this as objdir, from now on. The second line says, build me Thunderbird. The third line is because I need debugging printouts. The fourth line I just threw in for good measure. You can read more about configuration options here.
3) Build Thunderbird! "make -f client.mk build" (while still in src-thunderbird). This step will take a while (more than a half hour in my case).
4) Once the build is done, run Thunderbird with ./objdir/mozilla/dist/bin/thunderbird. Since we configured this build with the debug option, you will see a lot of printouts at your terminal.
Updating the repository
Make sure you're in the src-thunderbird directory. On the terminal, execute python client.py checkout. Re-build again.
Modifying the source files
The source files are everything outside your src-thunderbird file. Everything in it are your object files. When you modify your source files, you need to re-create your object files. You can run make -f client.mk build again (in your src-thunderbird dir), and it won't take as long as the first time.
If you changed files in only one directory, you can re-make only that directory. For example, if you changed files only in ~/src-thunderbird/mail/components, run make -C objdir/mail/components.
Getting distribution.ini working
1) This is a sample distribution.ini. This file must be placed in objdir/mozilla/dist/bin/distribution.
2) Add "distribution.js \" to the EXTRA_JS_MODULES section of Makefile.in (which resides in ~/src-thunderbird/mail/base/modules).
3) distribution.js should be in ~/src-thunderbird/mail/base/modules. I copied and modified the Firefox version. (Code referring to bookmarks and livemarks were ripped out.) I also changed the name of the object it exported to something like TBDistCustomizer.
(Note: JS modules have EXPORTED_SYMBOLS at the start of the file. This is as opposed to components files (which are also JS), which do not.)
4)
5) Modify mailGlue.js (in ~/src-thunderbird/mail/components) so that:
(a) It would import distribution.js (Components.utils.import("resource:///modules/distribution.js");); and
(b) it would create an instance of TBDistCustomizer. This intance can now be used to call the code in distribution.js that actually changes prefs (applyPrefDefaults).
So now I've come to the point where preferences set in distribution.ini are being set in Thunderbird, but there is a catch: some prefs are set properly, while others are duplicated. For example, mail.phishing.detection.enabled is set correctly, but something like network.cookie.lifetimePolicy instead becomes duplicated, with one version having the default and the other having the value that was set in distribution.ini.
I am still trying to figure out why this is so...
Climbing the brick wall
The problem was that distribution.ini doesn't like spaces around the "=" sign. network.cookie.lifetimePolicy = 1 should be network.cookie.lifetimePolicy=1.
The MailDev team at Mozilla! In particular Standard8 and sid0 for helping me get started.