RE: [Lse-tech] Re: Hugetlbpages in very large memory machines.......

From: Seth, Rohit <>
Date: 2004-03-16 10:31:23
>-----Original Message-----
>From: Ray Bryant
>Andrew Morton wrote:
><unrelated text snipped>
>> As for holding mmap_sem for too long, well, that can presumably be
>> around by not mmapping the whole lot in one hit?
>There are a number of places that one could do this (explicitly in user
>hidden in library level, or in do_mmap2() where the mm->map_sem is
>I'm not happy with requiring the user to make a modification to solve
>kernel problem.  Hiding the split has the problem of making sure that
if any
>of the sub mmap() operations fail then the rest of the mmap()
operations have
>to be undone, and this all has to happen in a way that makes the mmap()
>like a single system call.
>An alternative would be put some info in the mm_struct indicating that
>hugetlb_prefault() is in progress, then drop the mm->mmap_sem while
>hugetlb_prefault() is running.  Once it is done, regrab the
>clear the "in progress flag" and finish up processing.  Any other
>that got the mmap_sem and found the "in progress flag" set would have
>fail, perhaps with -EAGAIN (again, an mmap() extension).  One can also
>implement more elaborate schemes where there is a list of pending
>mmaps() with the associated address space ranges being listed; one
>check this list in get_unmapped_area() and return -EAGAIN if there is
>a conflict.

I think both of above options are bit of stretch.

>I'd still rather see us do the "allocate on fault" approach with
>to maintain the current ENOMEM return code from mmap() for hugepages.
Let me
>work on that and get back to y'all with a patch and see where we can go

I think this allocation on fault behavior will become essential when
Andi's mbind becomes part of the base kernel. And this scheme has an
added advantage of following normal semantics of page allocation (if a
user wants preallocation then MAP_LOCKED can be used).  As Andrew said
earlier in the thread that this though runs the risk of different
behavior with applications that currently assume pre-faulting behavior
in terms of performance (even if you decrement count upfront but do lazy
allocation).  As they will get penalized at fault time.  But this is the
kind of optimization that apps can do when porting to 2.6 based

> I'll start by taking a look at all of the arch dependent
hugetlbpage.c's and
>see how common they all are and move the common code up to
>(or did WLI's note imply that this is impossible?)

You should be able to move prefault code to common tree.

>However, is this set of changes something that would still be accepted
in 2.6,
>or is this now a 2.7 discussion?
>Best Regards,
>                   Ray Bryant
>512-453-9679 (work)         512-507-7807 (cell)
>The box said: "Requires Windows 98 or better",
>            so I installed Linux.
>To unsubscribe from this list: send the line "unsubscribe linux-ia64"
>the body of a message to
>More majordomo info at
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to
More majordomo info at
Received on Mon Mar 15 18:40:25 2004

This archive was generated by hypermail 2.1.8 : 2005-08-02 09:20:24 EST