[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [sup-devel] Heliotrope limitations for backward synchronization, , ,



Hi Horacio,

FWIW, you may use -- or even just take a look at -- the 'gmail' gem.
It works pretty well for back syncing and is built on RFCs too (if I'm not mistaken).
For maildir backsyncing, the 'maildir' gem also do the job,
both are using the great 'mail' gem (which I hope, Heliotrope & co will use soon).

Vivien.

On Mon, May 23, 2011 at 10:42 AM, Horacio Sanson <hsanson@gmail.com> wrote:
Now that I have GMail to Heliotrope initial synchronization (first
time) and incremental (new messages) synchronization working in my
script I started working on Heliotrope to GMail synchronization.
Unfortunately I found some difficulties to achieve this.

I am following rfc4549.txt and the client-to-server synchronization
says verbatim:

    c) "Client-to-server synchronization": for each IMAP "action" that
     was pending on the client, do the following:

     1) If the action implies opening a new mailbox (any operation that
        operates on messages), open the mailbox.  Check its UID
        validity value (see Section 4.1 for more details) returned in
        the UIDVALIDITY response code.  If the UIDVALIDITY value
        returned by the server differs, the client MUST empty the local
        cache of the mailbox and remove any pending "actions" that
        refer to UIDs in that mailbox (and consider them failed).  Note
        that this doesn't affect actions performed on client-generated
        fake UIDs (see Section 5).

     2) Perform the action.  If the action is to delete a mailbox
        (DELETE), make sure that the mailbox is closed first (see also
        Section 3.4.12 of [RFC2683]).


Seems simple to do but Heliotrope currently does not store/provide
enough information to implement this like:

 1)  Account and Mailbox information of messages.
 2) Heliotrope msg_id to GMail UID map.
 3) Per mailbox action FIFO queues.


Each action performed via Heliotrope (add label, remove label, state
change, delete) should be stored in some kind of FIFO associated to
the an account/mailbox pair. For example adding a label to a message
in my personal account's inbox would add an action like:

  label_add <label> <msg_id>

where label_add is the action, <label> the action parameter and
<msg_id> the message ID as seen by Heliotrope.  Then during
synchronization for an account I would iterate over all mailboxes
associated to the account and for each one I would replay the same
actions stored in the mailbox FIFO in the GMail server. Once an action
succeeds on the GMail side I  pop it out of the FIFO and continue with
the next action. Using the msg_id to UID map I can determine to what
message on the GMail side the action should be performed on.

Now how to implement these features? One way could be using the extra
parameters hash in the Heliotrope::Index::message_add method to store
the messages account, mailbox and UID's of each message when they are
added to Heliotrope index. This would also require a way to query this
information.

Another way would be to use special labels to mark mailboxes and
accounts. And add some new query formats to get all mailboxes for a
certain account and the FIFO actions of the mailbox.

Finally I am more inclined to keeping a separate Index/Store for each
account and access them via REST URLs like:

  POST  /<account>                                              #
Create new account Index/Store files
  POST   /<account>/<mailbox>                       # Create new
mailbox in account. Implemented using special labels?
  GET     /<account>/mailboxes                        # List all
mailboxes for account
  GET     /<account>/<mailbox>/action          # Get the next pending
action on mailbox
  DELETE /<account>/<mailbox>/action       # Delete next pending
action on mailbox
  POST  /<account>/<mailbox>/message     # Add new message to an
account/mailbox
  GET    /<account>/<mailbox>/message     # Get all messages from
account/mailbox

This of course is how I think  IMAP offline synchronization could be
done. If there are other ways to do this I am open to suggestions.

regards,
Horacio Sanson
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel



--
Vivien Didelot,
vivien.didelot.org
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel