Re: Balanced packing strategy

On Saturday 12 November 2005 14:59, Petr Baudis wrote:
> The repacking should be done in such a way to minimize the overhead for
> the dumb transport users. Ideal for this is some structure like (at the
> end of october):
> 
> 	year2003.pack
> 	year2004.pack
> ...
> 	week42.pack
> 	week43.pack
> 	<individual objects for weeks 43, 44>

I am not sure if it is really beneficial, as packs have the requirement
to be self contained, so you get a lot of objects undeltified which could
be deltified in a better scheme (as eg. in git native protocol).

AFAICS, the git native protocol (which is nothing more than a pack itself
for each transfer) even has this problem, too: If you are updating every
day via git native, the sum of transfered bytes in a month will be a
multiple of one git transfer for all the month's changes.

To keep the pack self-containment property, but work better with dumb
transfers, we could introduce incremental packs:

Instead of fully repacking, create a new pack by only appendending new
objects at the end of the pack. Thus, most objects will be appended in
deltified form, making the incremental addition quite small. The outcome
would be a totally new package.

Unfortunately, I do not know the package format in detail, and hope that
this is possible at all.

For dumb protocols to take advantage of this, the information that the
first part of a package is actually the same as another package has to
be stored somewhere visible.
If a client detects that it has the first part of a pack already locally,
it would be enough to fetch only some the second part. 

This is more or less the same as Pasky's solution, but by using incremental
packs instead. I think that such incremental packing will not even take
much more space that fully repacking.

Josef
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html