[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [sup-devel] [PATCH] XapianIndex.each_message_in_thread_for yields messages in cronological order

To: sup-devel <sup-devel@rubyforge.org>
Subject: Re: [sup-devel] [PATCH] XapianIndex.each_message_in_thread_for yields messages in cronological order
From: Tero Tilus <tero@tilus.net>
Date: Wed, 30 Dec 2009 04:41:02 +0200
Authentication-results: mx.google.com; spf=pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-devel-bounces@rubyforge.org
Delivered-to: eg@gaute.vetsj.com
Delivered-to: sup-devel@rubyforge.org
In-reply-to: <1261938751-sup-9421@zyrg.net>
List-archive: <http://rubyforge.org/pipermail/sup-devel>
List-help: <mailto:sup-devel-request@rubyforge.org?subject=help>
List-id: Sup developer discussion <sup-devel.rubyforge.org>
List-post: <mailto:sup-devel@rubyforge.org>
List-subscribe: <http://rubyforge.org/mailman/listinfo/sup-devel>, <mailto:sup-devel-request@rubyforge.org?subject=subscribe>
List-unsubscribe: <http://rubyforge.org/mailman/options/sup-devel>, <mailto:sup-devel-request@rubyforge.org?subject=unsubscribe>
References: <1261485246-sup-4236@tilus.net> <1261938751-sup-9421@zyrg.net>
Reply-to: Sup developer discussion <sup-devel@rubyforge.org>
Sender: sup-devel-bounces@rubyforge.org
User-agent: Sup/git

Rich Lane, 2009-12-27 23:37:
> Hmm. Threading should only depend on refs and reply-tos, not the date.

I think threading _should_ depend on date too.  Not of course the
parent-connections, but the ordering of siblings.  So even this bug(?)
aside the messages should afaik be processed in chronological order
when threading to get siblings ordered by date.

> Could you give a short example (just the relevant headers) of a
> situation where this patch helps?
> 
> What you describe sounds like a malformed message. What client is
> generating them / how common are they?

For what I know you might trigger this by replying to many messages at
once and thus having a list of ids in-reply-to header (in whatever
order of course, rfc doesn't require any particular order) instead of
one.  Then when you reply to this message using MUA that is bold
enough to try to form References: with the standard in-reply-to +
my-id method even if RFC 2822 says "trying to form a References: field
for a reply that has multiple parents is discouraged and how to do so
is not defined in this document".  You end up having References: which
has bunch of (thread-wise) random ids in random order instead of the
rfc-specified original, reply, replytoreply, etc. chain of ids.

Workaround is easy.  Just process messages sorted by date so the
in-reply-to fields of original messages override the fscked up
references of some latter mangled replies, which of course appear
_after_ any of the messages which threading they could possibly fsck
... they wouldn't be replies if they didn't. ;)

This thread was the itch that made me scratch.  I haven't really
looked for other twisted threads, but I've got several thousands of
mails from these same authors so I assume this is not singular case.
User agent headers also included.

Fscked up threading looks like this (produced by current git next)

+    Person Three, joulu 18 (2 weeks ago)
  +    Person Four, joulu 18 (2 weeks ago)
    +    Person Four, joulu 17 (2 weeks ago)
      +    Person One, joulu 17 (2 weeks ago)
        +    Person Five, joulu 17 (2 weeks ago)
          +    Person Four, joulu 17 (2 weeks ago)  
            +    Person Three, joulu 15 (2 weeks ago)  
              +    Person Two, joulu 15 (2 weeks ago)
                +    Person One, joulu 15 (2 weeks ago)  
  +    Person Four, joulu 18 (2 weeks ago)
    +    Person Three, joulu 18 (2 weeks ago)
  +    Person Two, joulu 18 (2 weeks ago)
    +    Person One, joulu 18 (2 weeks ago)
      +    Person One, joulu 19 (2 weeks ago)

Correct like this (produced by current git next + threading and date
format patches, and that's why date formats differ too)

+    Person One, 15. 12:38 (2 weeks ago)
  +    Person Two, 15. 14:17 (2 weeks ago)
  +    Person Three, 15. 14:35 (2 weeks ago)
  +    Person Four, 17. 01:47 (2 weeks ago)
    +    Person Five, 17. 02:28 (2 weeks ago)
    +    Person One, 17. 09:08 (2 weeks ago)
      +    Person Four, 17. 11:26 (2 weeks ago)
  +    Person Four, 18. 01:15 (2 weeks ago)
    +    Person Three, 18. 10:15 (2 weeks ago)
      +    Person Two, 18. 12:16 (2 weeks ago)
        +    Person One, 18. 13:30 (2 weeks ago)
          +    Person One, 19. 13:43 (2 weeks ago)
      +    Person Four, 18. 14:16 (2 weeks ago)
        +    Person Three, 18. 14:53 (2 weeks ago)

The headers in the order the messages appear in correct threading.

Date: Tue, 15 Dec 2009 12:38:28 +0200
From: Person One
Message-ID: <20091215103828.GA8328@domain-one>
User-Agent: Mutt/1.5.20 (2009-06-14)

Date: Tue, 15 Dec 2009 14:17:38 +0200
From: Person Two
Message-ID: <1260879458.2530.42.camel@havelock>
In-Reply-To: <20091215103828.GA8328@domain-one>
References: <20091215103828.GA8328@domain-one>
X-Mailer: Evolution 2.28.1

Date: Tue, 15 Dec 2009 14:35:01 +0200 (EET)
From: Person Three
Message-ID: <alpine.LRH.1.10.0912151434380.12088@domain-two>
In-Reply-To: <20091215103828.GA8328@domain-one>
References: <20091215103828.GA8328@domain-one>
User-Agent: Alpine 1.10 (LRH 962 2008-03-14)

Date: Thu, 17 Dec 2009 01:47:59 +0200
From: Person Four
Message-ID: <4B2971AF.7060808@domain-three>
In-Reply-To: <20091215103828.GA8328@domain-one>
References: <20091215103828.GA8328@domain-one>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
    rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0

Date: Thu, 17 Dec 2009 02:28:55 +0200 (EET)
From: Person Five
Message-ID: <alpine.DEB.2.00.0912170214460.25488@domain-five>
In-Reply-To: <4B2971AF.7060808@domain-three>
References: <20091215103828.GA8328@domain-one>
    <4B2971AF.7060808@domain-three>
User-Agent: Alpine 2.00 (DEB 1167 2008-08-23)

Date: Thu, 17 Dec 2009 09:08:31 +0200
From: Person One
Message-ID: <20091217070831.GD27029@domain-one>
In-Reply-To: <4B2971AF.7060808@domain-three>
References: <20091215103828.GA8328@domain-one>
    <4B2971AF.7060808@domain-three>
User-Agent: Mutt/1.5.20 (2009-06-14)

Date: Thu, 17 Dec 2009 11:26:15 +0200
From: Person Four
Message-ID: <4B29F937.7080909@domain-four>
In-Reply-To: <20091217070831.GD27029@domain-one>
References: <20091215103828.GA8328@domain-one>    <4B2971AF.7060808@domain-three>
    <20091217070831.GD27029@domain-one>
User-Agent: Thunderbird 2.0.0.23 (X11/20090817)

Date: Fri, 18 Dec 2009 01:15:33 +0200
From: Person Four
Message-ID: <4B2ABB95.6010301@domain-three>
In-Reply-To: <20091215103828.GA8328@domain-one>
References: <20091215103828.GA8328@domain-one>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
    rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0

Date: Fri, 18 Dec 2009 10:15:45 +0200 (EET)
From: Person Three
Message-ID: <alpine.LRH.1.10.0912181012570.30704@domain-two>
In-Reply-To: <4B2ABB95.6010301@domain-three>
References: <20091215103828.GA8328@domain-one>
    <4B2ABB95.6010301@domain-three>
User-Agent: Alpine 1.10 (LRH 962 2008-03-14)

Date: Fri, 18 Dec 2009 12:16:57 +0200
From: Person Two
Message-ID: <1261131417.2530.179.camel@havelock>
In-Reply-To: <alpine.LRH.1.10.0912181012570.30704@domain-two>
References: <20091215103828.GA8328@domain-one>
    <4B2ABB95.6010301@domain-three>
    <alpine.LRH.1.10.0912181012570.30704@domain-two>
X-Mailer: Evolution 2.28.1

Date: Fri, 18 Dec 2009 13:30:10 +0200
From: Person One
Message-ID: <20091218113010.GI3160@domain-one>
In-Reply-To: <1261131417.2530.179.camel@havelock>
    <alpine.LRH.1.10.0912181012570.30704@domain-two>
    <4B2ABB95.6010301@domain-three> <4B29F937.7080909@domain-four>
    <20091217070831.GD27029@domain-one>
    <alpine.DEB.2.00.0912170214460.25488@domain-five>
    <4B2971AF.7060808@domain-three>
    <alpine.LRH.1.10.0912151434380.12088@domain-two>
    <1260879458.2530.42.camel@havelock>
    <20091215103828.GA8328@domain-one>
User-Agent: Mutt/1.5.20 (2009-06-14)

Date: Sat, 19 Dec 2009 13:43:14 +0200
From: Person One
Message-ID: <20091219114314.GA15682@domain-six>
In-Reply-To: <20091218113010.GI3160@domain-one>
References: <alpine.LRH.1.10.0912181012570.30704@domain-two>
    <4B2ABB95.6010301@domain-three> <4B29F937.7080909@domain-four>
    <20091217070831.GD27029@domain-one>
    <alpine.DEB.2.00.0912170214460.25488@domain-five>
    <4B2971AF.7060808@domain-three>
    <alpine.LRH.1.10.0912151434380.12088@domain-two>
    <1260879458.2530.42.camel@havelock>
    <20091215103828.GA8328@domain-one>
    <20091218113010.GI3160@domain-one>
User-Agent: Mutt/1.5.18 (2008-05-17)

-- 
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel

Follow-Ups:
- Re: [sup-devel] [PATCH] XapianIndex.each_message_in_thread_for yields messages in cronological order
  - From: William Morgan <wmorgan-sup@masanjin.net>

References:
- Re: [sup-devel] [PATCH] XapianIndex.each_message_in_thread_for yields messages in cronological order
  - From: Rich Lane <rlane@club.cc.cmu.edu>

Prev by Date: [sup-devel] [PATCH 3/4] immediate thread indexing
Next by Date: [sup-devel] [PATCH] Make sup-tweak-labels work as advertised with no sources listed
Previous by thread: Re: [sup-devel] [PATCH] XapianIndex.each_message_in_thread_for yields messages in cronological order
Next by thread: Re: [sup-devel] [PATCH] XapianIndex.each_message_in_thread_for yields messages in cronological order
Index(es):
- Date
- Thread