[sigcomm] considerations for reviewing extended papers

[John Heidemann]

On Sun, 07 May 2006 12:50:22 PDT, Vern Paxson wrote: 
>> Synthesis should be a delta; it should describe how the pieces fit
>> together, in addition to just concatenating the pieces.
>> 
>> > I do think that a sizable chunk of any SIGCOMM
>> > paper should be new, unpublished work. 
>> 
>> If "sizeable", then we have a new criteria that argues against
>> publishing earlier, preliminary work in workshops ...
>
>It may be that in fact there's no significant disagreement here.
>If synthesis is indeed more than just concatenating the pieces, then
>I agree that the architectural framing is a contribution.  Similarly,
>I expect that "sizeable" will be in-the-eye-of-the-PC, and I imagine that
>the definition will necessarily be determined on a case-by-case basis.
>This is imperfect, but hard to remedy if we start (as I believe we should)
>with the qualitative notion that there needs to be a contribution beyond
>the workshop paper(s) in order for the prior publication not to be considered
>against the longer paper.

Vern, I heard you describe this at SIGCOMM, and now twice recently on
the mailing list, and I keep keep thinking I understand your point,
but then hearing something in it that doesn't make sense to me.

Let me try to decompose your paragraph above:

>It may be that in fact there's no significant disagreement here.
>If synthesis is indeed more than just concatenating the pieces, then
>I agree that the architectural framing is a contribution. 

If we call the early papers W1 and W2, and the new paper C, then I
take this as saying:

C must be W1 + W2 + X where X is something new

perhaps X is architectural framing, new application or experiment, or something

>Similarly,
>I expect that "sizeable" will be in-the-eye-of-the-PC, and I imagine that
>the definition will necessarily be determined on a case-by-case basis.
>This is imperfect, but hard to remedy if we start (as I believe we should)
>with the qualitative notion that there needs to be a contribution beyond
>the workshop paper(s)...

And what you say here is that the necessary size of X must be
determined by the PC.  But X > 0.

This far, I can fully agree with this policy.  And I don't think it
represents a policy different than any typical PC or journal.
(And ACM and IEEE guidelines that every paper has to have something
new about it.)

But then you add:

>... in order for the prior publication not to be considered
>against the longer paper.

Or your post that sparked this round of discussion, you said
"...SIGCOMM [should] treating earlier, short papers as the same
as technical reports."

AFAIK everyone regards tech reports as non-publications, so if W1 and
W2 are treated as tech reports and are "not to be considered against"
C, then the size of X can be 0.

(Since if X must be larger-than-0, then W1 and W2 ARE being held
against C, in proprotion to the required size of X.)

So I'm confused... how am I misreading what you've written?  Or if
not, which policy are you suggesting SIGCOMM adpot?  X>0, or X>=0?


The only way I can make sense of what you're saying is to think that
"not being held against" means that X can be quite small.  I
personally think, for conferences, that this must be left to the PC
and PC chairs.  (Which, actually, you say, too...)

I can offer one datapoint, however.  The rule-of-thumb for IEEE
journal submissions that I was told (by an EIC) is X must be at least
20-30% of the journal submission.  For better or worse, the current
journal webpage doesn't quantify this, but simply says X must be
"substantial".

These targets sound reasonable to me for our community's journal-like
conferences, provided the ultimate determination is made by the PC.
Which, I think it clearly must be, since I think it's pretty hopeless
to quantify "contribution".

   -John