Migrating to Google Apps

From ZaberWiki

Jump to: navigation, search

This article is unrelated to Zaber Products but has been useful to us internally so I thought I would post it online for everyone. We've recently decided to migrate Zaber's internal email server to Google Apps. This article at lifehacker.com was an invaluable resource for setting up Thunderbird client to interface with Google mail. Some of the information below can also be found there, though I've added many personal notes. The following is a description of the process (which is ongoing). - Rob 14:28, 19 January 2008 (PST)

Contents

[edit] What is Google Apps

Google apps is a feature of google that allows users to log into various google applications (ex gmail, calendar, and docs) using an email address with a non-google domain (for example, you can use gmail for email even though your email address is xxxx@some_domain.com, and the people you correspond with will see no indication you are using gmail). Additionally Google Apps can automatically set up users on the same domain for online collaboration (documents and calendars can be shared between users, and all users on the same domain are added to each other's chat and email contacts by default).

[edit] Migration to Google Apps

[edit] Why migrate to Google Apps

The primary reasons we are migrating to Google Apps (and why I'm writing this) are:

  1. Google has the best spam detection around (it's basically human detection based on their entire user base... if 1 million people think something iss spam, it probably is).
  2. Google allows automatic alias creation (name+XXX@some_domain.com is an alias of name@some_domain.com)
  3. Google just implemented IMAP so Thunderbird can be used as an email client to work with multiple GMail accounts
  4. Google offers over 6 Gigs of email storage per user
  5. Gmail organizes mail by conversations and allows you to to a google search on your email archive
  6. Google allows users to set their own auto-responders without administrator access
  7. Google Apps allows simultaneous editing of shared documents and spreadsheets by members of your domain
  8. Google Apps is a free service for up to 25 users, and rates are reasonable above that

Other services can compete on some of those points, but I have yet to see any that come close overall.

[edit] What's involved in migrating to Google Apps

The final step in the migration process is to change the MX server at your Domain Host to point to Google's Server. But before that, it's possible to gradually migrate individual user email accounts to Google Apps for testing purposes. All google apps accounts have a domain alias that is functional whether your MX server points to Google or not. For example, if you set up a google apps account for the domain your_domain.com and set up a user admin, then you can simply set the admin@your_domain.com account on your current email server to forward all mail to admin@your_domain.com.test-google-a.com. The following table illustrates how email flow will work during and after migration: 

 User Account Status       DomainHost      Old Email Server         Google Server          User's Computer
During Migration:
 Non-migrated user ------> redirect -----> stored ---------------------------------------> webmail/POP/IMAP to Old Server
 Migrated user ----------> redirect -----> forward ---------------> stored --------------> webmail/IMAP to Google
After Migration:
 Migrated user ----------> redirect ------------------------------> stored --------------> webmail/IMAP to Google

Here's a summary of the steps involved in migrating an individual account:

  1. Check for any aliases used on the current user account. Usually there are none, but if there are, contact the necessary parties and change the aliases to the name+alias@your_domain.com format. Alternatively you can add the aliases as "nicknames" to the new user in Google Apps, but the user won't have access to change them (only Administrators have access to "nicknames".
  2. Add the user to Google Apps (this automatically creates a Gmail account for them) and add any nicknames you want
  3. Set the account on the old server to forward to Google (user@your_domain.com.test-google-a.com).
  4. Log in to the users gmail account, click "settings", then "forwarding and POP/IMAP" and enable IMAP (and POP if you want)
  5. Upload the user's old email into their new Gmail account (Use Thunderbird with an IMAP connection and just drag and drop old email into the new folders... more details on this below).
  6. Set up the user's email client (Thunderbird instructions below) if they use one.
  7. Have the user login and change their password.

[edit] Using the Gmail Web interface after migration

After an account has been migrated the user can assess their email at: 

http://mail.google.com/a/your_domain.com    (where your_domain.com is your own domain... duh)

You can also set your Domain Host to redirect to Google from a more convenient URL like: 

http://gmail.your_domain.com

See the Google Apps control panel for more details on that.

Users can change their password by clicking "Settings" in the top right, then "Accounts", then look for the "Change Password" link at the bottom.

[edit] Using Gmail IMAP interface with Thunderbird 2.X

Thunderbird can be used to connect to Gmail accounts by IMAP which allows you to view the email stored on the Google server without downloading it (unless you want to) to your own computer. If you have only 1 account, it's easier just to use webmail, but if you need to monitor multiple email accounts, then Thunderbird is worth setting up.

[edit] How some functions map between Thunderbird 2.X and Gmail

Gmail uses labels, while Thunderbird uses folders. Therefore there are some subtleties when mapping actions between the two. The table below shows the default behavior. Changing settings in Thunderbird is required to improve this behavior. 

Thunderbird Action       -->      Gmail Equivalent
Open a message                    Mark a message as read
Flag a message                    Apply a star to the message
Create a folder                   Create a label
Move a message to "folder"        Remove the current label and apply the label "folder"
Move a message to [Gmail]/Spam    Remove all labels and report a message as spam 
                                  (also removes it from [Gmail]/All Email and deletes it "forever" after 30 days)
Move a message to [Gmail]/Trash   Remove all labels and apply the Trash label 
                                  (also removes it from [Gmail]/All Email and deletes it "forever" after 30 days)
Send a message                    Store message in [Gmail]/Sent Mail (if you used the gmail smtp server)
                                  and in [IMAP]/Sent Mail (Thunderbird default)
Delete a message                  Remove the current label and add [IMAP]/Trash label
                                  (does not actually delete the message or remove it from [Gmail]/All Email)

[edit] Setting up Thunderbird to access a Google Apps gmail account with IMAP

I'll use the account user@your_domain.com as an example.

  • set up user@your_domain.com as a user in Google Apps, add nicknames, login to their gmail account and enable IMAP
  • if you haven't yet redirected your domain host's MX server to point to Google, set your current email server to forward to user@your_domain.coom.test-google-a.com.
  • set up "user@your_domain.com (Gmail)" as an IMAP account in Thunderbird
    • imap server: imap.gmail.com
    • username: user@your_domain.com
  • In Account Settings
    • Change outgoing SMTP server to smtp.gmail.com. You may need to create a new smtp entry first (use TLS, port 587)
    • Connect to IMAP server using SSL, port 993
    • Select "When I delete a message" to "Remove it immediately" (this will simply remove the label, so the email will still be accessible in [Gmail]/All Mail). This prevents Thunderbird from creating its own Trash folder and moving deleted messages there. I highly recommend against it, but it is possible to later set Thunderbird to move deleted messages to [Gmail]/Trash. This is described below, but for now you should just set thunderbird to remove messages immediately.
  • Close Account Settings and select the inbox of the new account in the folder pane
  • Enter the password (whatever it may be) and you should see the [Gmail] folder
  • Sync Trash folder (This is not recommended, but you can decide for yourself. See below)
  • Sync Sent mail folder (see below)
  • Sync Drafts folder (see below).
  • Install some useful extensions:
If you have multiple accounts, install the Folderpane Tools extension to change the order they appear in the folder pane
If you don't store any mail in Local Folders, install the TweakMail extension to remove "Local Folders" from the folder pane
If you want to search your email using similar syntax to Gmail, install the GmailUI extension (Set its archive to [Gmail]/All Mail)

[edit] Notes on Conversations vs Individual Messages

Gmail organizes emails in conversations. Once Gmail decides messages are part of the same conversation, those messages appear together ALWAYS. In Gmail, new messages that arrive in your inbox show up attached to the conversation they are a part of (even if the conversation has been archived or has a different label. In Thunderbird, each message is it's own entity with it's own label (the name of the folder in which it resides). Two messages can be part of the same conversation but be located in two different folders. In Gmail, clicking either label will show the whole conversation, but in Thunderbird, clicking either folder will only show the message that is stored in that folder.

For example: Suppose you have messages A,B, and C that are part of the same conversation in your inbox. In Thunderbird, you move each message from the inbox to it's own folder FolderA, FolderB, FolderC. As far as Gmail is concerned, the "Conversation" now has 3 different labels. As far as Thunderbird is concerned, each message has only one label (its own folder name). Now suppose you receive a new message D in your inbox that is part of the same conversation. Gmail will show the full conversation (4 messages) in your inbox with 3 different labels (FolderA, FolderB, FolderC). Thunderbird, however, will still only show one message in each folder, and the new message D in the inbox. Now here's the confusing part. If you log into Gmail and archive the new message, then as expected, in Gmail, the conversation no longer shows up in the inbox, but still shows up for each of the 3 labels (FolderA through FolderC). But what happens in Thunderbird? The new message D simply disappears from the inbox (though it's still available in [Gmail]/All Mail), and each of the three folders still only contains one email. Now suppose you take the conversation in Gmail and add a new label, FolderD, to it. In Thunderbird you will see a new folder containing all 4 messages.

As shown above, it is impossible to tell in Gmail whether a "conversation" with a given label means that all, some, or just one of the messages in the conversation has that label. Therefore, if you use Thunderbird to access your google mail via IMAP, it is a good idea not to label or archive conversations using the webmail interface as it's difficult to predict what effect you'll be having on the Thunderbird side of things. It's safe to label or archive individual messages. Just not conversations. However, as a general rule of thumb, if you're going to use Thunderbird (or any email client for that matter) as your primary email access then if you do occasionally use the webmail interface, you shouldn't label or archive anything.

Because this problem manifests when messages conversation split between different folders, it's a good idea in Thunderbird to keep messages that are part of the same conversations together in the same folder. This also allows you to use the threading feature in Thunderbird and see both sent messages and replies in the same thread.

[edit] How to sync the trash folder

This is a way to set Thunderbird to move deleted email to [Gmail]/Trash instead of [IMAP]/Trash (the default). Following the instructions above, deleted email will not actually be deleted. It will just have its label removed.

Warning: Gmail only stores one copy of any email but it may have multiple labels and could therefore show up in multiple folders in Thunderbird. After making the changes below, deleting a message will move it to the [Gmail]/Trash folder. However, the message may still exist in other folders. When Gmail deletes the message from the [Gmail]/Trash folder after 30 days, it will remove that message irretrievably from any other folders it happens to be in too. So be sure you know what you are doing if you sync the trash folder. If you ever attach multiple labels to the same message, or copy the same message to multiple folders, you should not sync the trash folder.

A safer option besides syncing the trash folder is to do as described in the above section (simply set deleted messages to be removed immediately rather than moved to Trash). Then deleting in Thunderbird will only remove the current folder label, leaving others (if there are any) intact. You can "delete forever" by moving a message to the [Gmail]/Trash folder and then deleting it. Email deleted from [Gmail]/Trash or [Gmail]/Spam will be completely deleted (assuming thunderbird isn't set to move deleted emails anywhere else).

But if you really want to know that you've deleted an email when you hit the delete key, here's how to do it:

  • tools/options/config editor...
  • enter "server.server" in filter field and find the server number X in mail.server.serverX where X corresponds to the desired account
  • right click and select New/String
  • enter the preference name mail.server.serverX.trash_folder_name (where X is the number determined above) and click OK
  • enter the string "[Gmail]/Trash" and click OK
  • in account settings, choose Select "When I delete a message" to "Move it to the Trash folder"
  • close thunderbird
  • log into the gmail account through the web interface and delete the [IMAP]/Trash label if it exists.
  • restart thunderbird, delete an email from your account and ensure it goes to the [Gmail]/Trash folder. restart thunderbird again, and the folder icon should have changed to the trash can you are used to.

[edit] How to sync the sent mail folder

  • by default, thunderbird uses it's own "sent mail" folder
  • tools/account settings.../<account name>/Copies & Folders
  • if you send mail using smtp.gmail.com and log in using the correct account name, then gmail will automatically store sent messages in the [Gmail]/Sent Mail folder. Under "When sending message, automatically:" you can unselect "Place a copy in:". If you're like me and prefer to keep sent email in your inbox so you can view complete threads in Thunderbird, then specify to keep a copy there, but keep in mind that a copy will still be put in [Gmail]/Sent Mail on whatever account you used to login to smtp.gmail.com.
  • If you use a different smtp server to send mail, then you have to specify where to store the sent mail. Under "When sending message, automatically:" check "Place a copy in:", select "other" and specify where you want to store sent mail. Again, I recommend puting it in your inbox so you can view complete threads in Thunderbird.
  • In either case, after restarting Thunderbird, you should be able to delete Thunderbird's default Sent Mail folder. If not, close Thunderbird, log in to Gmail, and delete the [IMAP]/Sent Mail folder there. Then restart Thunderbird.

Note: It's important that you do not use smtp.gmail.com as the smtp server for accounts other that the one you use to log into the smtp server. If you do, gmail will automatically store all outgoing messages in the "Sent Mail" folder on that account. If you have multiple Gmail accounts in Thunderbird, each one needs to use it's own login to smtp.gmail.com

[edit] How to set the drafts mail folder

Warning!!! - saving drafts from Thunderbird to any folder on google has the annoying effect that multiple copies of the same email (at progressive stages of completion) will be saved. Only the final version will be removed from the drafts folder when you finally send the message, leaving a bunch of garbage messages in the drafts folder that must be deleted manually. For this reason I think it's better to keep drafts stored in Local Folders/Drafts. Future versions of Thunderbird may correct this issue.

  • by default, thunderbird uses it's own "[IMAP]/Drafts" folder which it will create unless you change the settings.
  • tools/account settings.../<account name>/Copies & Folders
  • Under "When sending message, automatically:" check "Keep message drafts in:", select "other" and specify the new location 
Recommended: Local Folders/Drafts
Also OK:     personal_account_name/[Gmail]/Drafts. (But see Warning above)
Not OK:      shared_account_name/[Gmail]/Drafts. (You should use the drafts folder on your personal account)

[edit] Some advanced settings

You can use Thunderbird Configuration Editor (Tools/Options/Advanced/Config Editor...) to change a number of settings that cannot be changed through any other means.

[edit] mail.check_all_imap_folders_for_new

 true            = Thunderbird checks all folders for new messages.
 false (default) = Thunderbird checks the inbox only for new messages.

Set this to true if your Gmail account is set to automatically label all incoming messages. If only specific folders in addition to the inbox need to be checked, just right click those folders and select Properties/General Information/Check this folder for new messages. Note that this just determines what folders are checked for new messages. Other settings affect what Thunderbird does when it finds new messages.

[edit] mail.imap.use_status_for_biff

 true (default)  = Thunderbird uses the IMAP STATUS command to check the number of messages in folders.
                   This returns a message count for the folder, which Thunderbird uses to detect new mail.
 false           = Thunderbird uses the IMAP SELECT command to check the actual messages in folders.
                   This returns message headers, which Thunderbird stores in addition to detecting new mail.

Set this to false if you want Thunderbird to automaticlally download headers for new messages. Otherwise, the headers won't be downloaded until you select the folder.

Note: "Biff" was/is a mail notification system for Unix. It alerts the recipient when new mail arrives via a message on the user's terminal. In the context of Thunderbird, "biff" simply means "detection of new mail".

[edit] mail.server.default.download_on_biff

 true            = Thunderbird will automatically download new message bodies (and attachments).
*false (default) = Thunderbird won't download message bodies till you select the message.

Set this to true if you want Thunderbird to download entire message bodies when it detects new messages on the server. However, this will only work for folders that have been selected for offline use. To select folders for offline use, click tools/account settings/offline & disc space/select folders for offline use.

[edit] Using Gmail IMAP interface to set up a shared account in Thunderbird 2.X

I have experimented with having a "shared inbox" accessed by IMAP that multiple people can view and manage. Suppose for example you have a company email contact@your_domain.com and you want multiple user's to be able to view incoming messages. You could create a distribution list and have the messages sent to multiple people, but then you're duplicating email unnecessarily on multiple computers and other users will have no idea when someone has already responded to an incoming message. There are sophisticated email managment applications to deal with this issue, but it can also be resolved reasonable well simply by using IMAP connections from multiple computers to the same Gmail account. Since everyone is viewing the same email messages stored on Google's server, whatever one person does to those emails and folders will be reflected on everyone's views. Here are a few things to keep in mind:

[edit] Partial deletion

Don't sync the trash folder on shared inboxes. This makes it more difficult for anyone to permanently delete an email before someone else has a chance to see it. Without syncing the trash folder, deleting simply remove the email's label. The message will still be accessible through [Gmail]/All Mail. Users can check that folder from time to time in case anyone inadvertently deletes mail that is relevant to someone else. Letting people delete emails before others see them is generally a good thing (in the case of spam or irrelevant messages) but can also be a bad thing (if people delete email that is relevant to someone else). Some retraining of habits may be required. So far it has worked very well for us.

[edit] Permanent deletion

If users delete mail from [Gmail]/All Mail, or [Gmail]/Spam, or [Gmail]/Trash, then it will be permanently deleted. User's should not do this unless they are 100% sure NOBODY will want to see the email. With multiple people managing the same inbox, it's safest if no email is ever completely deleted.

[edit] Read/Unread status

Messages will be marked as "read" the first time ANYONE views them. This can be confusing if you use the read/unread status to tell which emails still need attention. There are a couple options to resolve this:

Option 1 is to change the "Wait ____ seconds before marking a message as read" setting on each Thunderbird installation (see tools/options/advanced). I set mine to 10,000 seconds (I would turn the feature off completely if it were an option). I find it much more intuitive to manually mark messages as read since what "read" means to me is "no longer needs my attention" which isn't necessarily true the instant I click on the message header. However, you don't have much control over what each user might change their settings to, so option 2 may be a better solution.

Option 2 is to set the Gmail account to automatically star messages as they arrive. Setting up a filter to do this is relatively easy. If the message Doesn't Have "AddAStarToAllIncomingMail" then Star it. Since it's pretty rare that anyone would send a message containing "AddAStarToAllIncomingMail", this filter will add a star to all incoming messages. Thunderbird versions prior to 2.0.0.9 use Flags instead of stars, but they behave the same. You need to enable the Flag or Star column to display by clicking on the little icon at the right of the column headers. Now users can assume any starred email hasn't been dealt with yet. If a user deals with an email (ex responds from their personal account) they can remove the star by clicking on it.

[edit] Spam

Gmail's spam filters will delete any obvious spam before you even see it. More questionable spam will be moved into the [Gmail]/Spam folder and automatically deleted after 30 days. Users should check there occasionally. If a user encounters spam in the inbox, they can move it into the [Gmail]/Spam folder to help Google learn (Google's spam filters are based on millions of users indicating what they think is spam and what isn't). Likewise if you encounter non-spam in the spam folder, move it into the inbox (and star it if it hasn't been dealt with yet).

[edit] Old mail

Periodically any user can delete old (unstarred) emails. If you've set up Thunderbird as recommended, this doesn't actually delete them. You will still have a complete log of all email into the shared inbox accessible in the [Gmail]/All Mail folder.

[edit] Replies

A shared inbox is not for replying, or storing replies. It's simply for multiple users to monitor the same inbox. Users should reply from their personal email accounts, and store their sent mail on their own account. To avoid accidently replying from the inbox account, its best to set up a dummy smtp server (that won't work) for that account.

[edit] Some issues to be aware of

  • Although possible, uploading emails to gmail through IMAP (by drag and drop in Thunderbird) is not fully supported. See http://mail.google.com/support/bin/answer.py?answer=81312&topic=12815. You will occasionally encounter a message that does not upload and you may have to try multiple times or skip the message entirely.
  • Dragging and dropping email into Gmail through IMAP does not always preserve the receipt date. As of Dec 4, 2007 this is a known issue that Google is trying to resolve. See http://mail.google.com/support/bin/answer.py?answer=82365&topic=12816. Gmail will sometimes display the date of the upload instead of the date of the original email which compromises usefulness of the search capabilities through the web interface. Thunderbird will still display the correct date though. Google is working to resolve this. Followup: In late 2007 I experienced this problem continually, but I have not had any problems since Jan, 2008. It appears that Google may have resolved this problem.
  • Dragging and dropping folders into Gmail sometimes only works for one level deep hierarchy. You may need to recreate the folder structure before copying emails over. Also, dragging and dropping an entire folder seems to perform a "copy" instruction while dragging and dropping individual emails seems to perform a "move" instruction (deleting the old emails if the move is successful).
  • If you delete an email from a Gmail account by moving it to the [Gmail]/Trash folder, then gmail will ignore any attempt to re-upload that same email by POP or IMAP. ie if you are using a gmail account to archive email, then you cannot re-upload any email that you have previously deleted from the gmail account.
Retrieved from "http://www.zaber.com/wiki/Migrating_to_Google_Apps"
Personal tools
Zaber Website