The ToDo list for Paul Davies

26/04/07 - 27/04/07

11/04/07 - 14/04/07

04/04/07 - 05/04/07

03/04/07

02/04/07

26/03/07 - 30/03/07

21/03/07

19-03-07

08/03/07

06/03/07

05/03/07

02/03/07

27/02/07 - 01/03/07

26/02/07

21/02/07

20/02/07

16/02/07 - 18/02/07

15/02/07

14/02/07

13/02/07

12/02/07

09/02/07

07/02/07 - 08/02/07

06/02/07

26/01/07 - 06/02/07

25/01/07

22/01/07

14/01/07

13/01/07

12/01/07

11/01/07

10/01/07

09/01/07

08/01/07

07/01/07

06/01//07

05/01/07

04/01/07

03/01/07

02/01/07

Prior to LCA I need to (preferably before 2 January)

  1. Implement a number of iterators
  2. Pretty up GPT formatting
  3. Make sure GPT survives LTP
  4. Make sure I can run SPEC and lmbench on top of GPT

January 2 - 15

  1. Feed PTI/GPT to linux-mm on the 7th
  2. Benchmark PTI on i386/IA64
    • would like to benchmark on ppc and x86-64 as well
  3. Benchmark GPT
  4. Do out a blow by blow description of the 50 odd patches and update the LCA for PTI page
  5. Prepare for 5 minute lightening talk at LCA (hopefully)

Time permitting

23/12/06

18/12/06 - 24/12/06

15/12/06

12/12/06

08/11/06

05/11/06

04/11/06

29/11/06

28/11/06

27/11/06

13/11/06

12/11/06

11/11/06

10/11/06

08/11/06

TODO - over the next week or so

07/11/06

06/11/06

03/11/06

02/11/06

01/11/06

31/10/06

30/10/06

26/10/06

25/10/06

24/10/06

23/10/06

19/10/06

17/10/06

16/10/06

12/10/06

11/10/06

10/10/06

06/10/06

05/10/06

04/10/06

03/10/06

02/10/06

29/09/06

Benchmarking results for ICE talk

***NO PTI***
2.6.17.2 vanilla VHPT OFF, 4 level page table, 16K page
                 null     null                       open    signal   signal    fork    execve  /bin/sh
kernel           call      I/O     stat    fstat    close   install   handle  process  process  process
------------  -------  -------  -------  -------  -------  -------  -------  -------  -------  -------
2.6.17.2        0.323  0.56164    2.794    0.711    5.613    0.906    4.058    154.1    787.7   4049.4
 s.d. (10 runs) 0.000  0.00058    0.005    0.002    0.023    0.001    0.015      0.0     13.9     30.5

             0K       0K       1K       1K       4K       4K      10K      10K     Mmap     Prot    Page
kernel     Create   Delete   Create   Delete   Create   Delete   Create   Delete   Latency  Fault   Fault 
-------- -------  -------  -------  -------  -------  -------  -------  -------  -------  ------  ------
2.6.17.2   55.85    24.28    84.09    42.19    87.35    42.05   123.83    45.02   5806.7   1.715    2.00
 s.d.       0.06     0.10     0.52     0.20     0.88     0.22     4.29     0.94    103.4   0.032    1.15

***PTI***
                  null     null                       open    signal   signal    fork    execve  /bin/sh
kernel            call      I/O     stat    fstat    close   install   handle  process  process  process
-------------  -------  -------  -------  -------  -------  -------  -------  -------  -------  -------
2.6.17.2         0.323  0.57173    2.794    0.705    5.573    0.907    4.087    157.1    778.5   4090.8
  s.d. (10 runs) 0.000  0.00018    0.008    0.000    0.021    0.001    0.010      0.0     20.2     17.1

            0K       0K       1K       1K       4K       4K      10K      10K     Mmap     Prot    Page
kernel   Create   Delete   Create   Delete   Create   Delete   Create   Delete   Latency  Fault   Fault
-------- -------  -------  -------  -------  -------  -------  -------  -------  -------  ------  ------
2.6.17.2  56.07    24.24    84.30    42.16    87.02    42.13   122.16    45.39   5978.1   1.674    1.90
  s.d.     0.08     0.11     0.78     0.20     1.08     0.23     4.71     0.14    114.0   0.023    0.32

SUMMARY
2% Slow down on fork
exec and page fault slightly quicker

***NO PTI***
2.6.17.2 vanilla VHPT ON, 4 level page table, 16K page

kernel           call      I/O     stat    fstat    close   install   handle  process  process  process
------------  -------  -------  -------  -------  -------  -------  -------  -------  -------  -------
2.6.17.2        0.323  0.56145    2.769    0.712    5.695    0.907    4.090    145.9    769.7   3990.7
 s.d. (10 runs) 0.000  0.00022    0.007    0.001    0.017    0.000    0.017      0.0     24.9     16.7


            0K       0K       1K       1K       4K       4K      10K      10K     Mmap     Prot    Page
kernel     Create   Delete   Create   Delete   Create   Delete   Create   Delete  Latency  Fault   Fault 
--------  -------  -------  -------  -------  -------  -------  -------  -------  -------  ------  ------
2.6.17.2    56.08    24.41    83.91    42.21    86.54    42.24   119.25    45.50   4917.1   1.618    1.50
  s.d.       0.06     0.09     0.75     0.18     1.09     0.22     3.05     0.11     41.4   0.041    1.58

***PTI***

                 null     null                       open    signal   signal    fork    execve  /bin/sh
kernel           call      I/O     stat    fstat    close   install   handle  process  process  process
         ----  -------  -------  -------  -------  -------  -------  -------  -------  -------  -------
2.6.17.2         0.323  0.57058    2.819    0.705    5.632    0.906    4.111    151.5    780.6   3984.3
 s.d. (10 runs)  0.000  6.74948    0.009    0.000    0.029    0.001    0.007      0.0     14.3     16.9

             0K       0K       1K       1K       4K       4K      10K      10K     Mmap     Prot    Page
kernel    Create   Delete   Create   Delete   Create   Delete   Create   Delete   Latency  Fault   Fault
-------- -------  -------  -------  -------  -------  -------  -------  -------  -------  ------  ------
2.6.17.2   56.18    24.53    84.84    42.35    87.29    42.36   119.74    45.55   5024.8   1.576    1.30
  s.d.      0.06     0.04     0.27     0.15     0.22     0.19     2.58     0.28     39.9   0.027    0.95

SUMMARY
3.8% fork slow down
exec slightly slower
page fault slightly quicker

28/09/06

27/09/06

26/09/06

25/09/06

21/09/06

20/09/06

19/09/06

18/09/06

15/09/06

14/09/06

11/09/06

08/09/06

06/09/06

01/09/06

31/08/06

30/08/06

29/08/06

28/08/06

22/08/06

21/08/06

18/08/06

17/08/06

16/08/06

12/08/06

11/08/06

10/08/06

09/08/06

08/08/06

07/08/06

04/07/06

02/07/06

01/07/06

31/06/06

28/06/06

27/06/06

25/06/06

24/06/06

22-23/06/06

20/06/06

19/06/06

18/06/06

17/06/06

14/07/06

13/07/06

10/07/06 - 11/07/06

09/07/06

08/07/06

07/07/06

06/07/06

05/07/06

04/07/06

03/07/06

02/07/06

01/07/06

30/06/06

28/06/06

24-27/06/06

23/06/06

22/06/06

21/06/06

20/06/06

18/06/06

16/06/06

15/06/06

14/06/06

13/06/06

11/06/06

10/06/06

08/06/06

07/06/06

06/06/06

05/06/06

30/05/06

29/05/06

24/05/06

23/05/06

22/05/06

19/05/06

18/05/06

17/05/06

16/05/06

15/05/06

07/05/06

06/05/06

05/05/06

04/05/06

03/05/06

02/05/06

01/05/06

28/04/06

27/04/06

26/04/06

25/04/06

24/04/06

18/04/06

17/04/06

14/04/06

13/04/06

I am in Queensland from 7/06/06 to 13/06/04 and consequently unavailable.

06/04/06

05/04/06

04/04/06

03/04/06

31/03/06

28/03/06

27/03/06

26/03/06

08/03/06

02/03/06

01/03/06

27/02/06

18/02/06

17/02/06

16/02/06

struct gpt_read_stack {
        GPTKey position;        /* Position in trie     */
        GPTNode *node;          /* Node                 */
        int pos;                /* Position in level    */
        Log2Size level_size;    /* Level size           */
        int pop;                /* To pop or not to pop */
};

static inline int page_table_read_iterator(struct mm_struct *mm,
        unsigned long addr, unsigned long end, pte_callback_t func, void *args)
{
    struct gpt_read_stack stack[GPT_READ_STACK];
    struct gpt_read_stack *sp=&stack[1];
    GPTKey temp, guard, level;
    GPTKey start_coverage, end_coverage, coverage;
    GPTKey key, end_key;
    unsigned long address;
    int err=0;

    (sp-1)->pop = 1;
    sp->pos = 0;
    sp->pop = 0;
    sp->level_size = 0;
    sp->node = &mm->node;
    gptKeySet(0,0, &sp->position);
    gptKeySet(extract_key(addr), GPT_KEY_LENGTH_MAX, &key);
    gptKeySet(extract_key(end),  GPT_KEY_LENGTH_MAX, &end_key);
    spin_lock(mm->page_table_lock);
    for(;;) {
        /* pop the stack - prevent underflow - */
        while(sp->pop) if(--sp==stack) return 0;

        /* Get the start and end coverage of our position in the trie */
        gptKeySet(sp->node->internal.guard_value, sp->node->internal.guard_length, &coverage);
        gptKeysMergeLSB(sp->position, &coverage);
        start_coverage = coverage;
        start_coverage.value <<= GPT_KEY_LENGTH_MAX - coverage.length;
        start_coverage.length = GPT_KEY_LENGTH_MAX;
        end_coverage = start_coverage;
        end_coverage.value += ((1UL<<(GPT_KEY_LENGTH_MAX - coverage.length)) - 1);

        /* Ensure we go to the right initialisation point */
        if(key.value <= start_coverage.value)
            key = start_coverage;

        /* Iterated past our region */
        if(key.value >= end_key.value) return 0;

        switch(sp->node->raw.type) {
        case GPT_NODE_TYPE_LEAF:
            gptKeySet(sp->node->leaf.guard_value,
            sp->node->leaf.guard_length, &temp);
            gptKeysMergeLSB(sp->position, &temp);
            address = get_real_address(temp.value);
            if(address>=addr)
                if(err = func(mm,&(sp->node)->leaf.payload, address, args)) return err;
            sp->position.value++;
            sp->node++;
            sp->pos++;
            sp->pop = 0;
            break;

        case GPT_NODE_TYPE_INTERNAL:
            if((key.value >= start_coverage.value) && (key.value <= end_coverage.value)) {
                sp++; /* PUSH STACK */
                gptKeySet((sp-1)->node->internal.guard_value,
                     (sp-1)->node->internal.guard_length, &guard);
                gptKeySet(0x0, (sp-1)->node->internal.level_log2size, &level);
                merge_3keys((sp-1)->position, guard, level, &sp->position);
                sp->pos = 0;
                sp->node = (GPTNode *)__va((sp-1)->node->internal.level_ptr);
                sp->level_size = ((1<<((sp-1)->node->internal.level_log2size))-1);
                sp->pop = 0;
                (sp-1)->position.value++;
                (sp-1)->pos++;
                (sp-1)->node++;
                if((sp-1)->pos > (sp-1)->level_size) (sp-1)->pop = 1;
            } else {
                sp->position.value++;
                sp->node++;
                sp->pos++;
                sp->pop=0;
                if(sp->pos>sp->level_size) sp->pop = 1;
            }
            break;

        case GPT_NODE_TYPE_INVALID:
            sp->position.value++;
            sp->node++;
            sp->pos++;
            sp->pop = 0;
            if(sp->pos>sp->level_size) sp->pop = 1;
            break;

        }
    }
    spin_unlock(mm->page_table_lock);
    return err;
}

08/02/06

06/02/06

31/01/06

26/01/06

25/01/06

25/01/06

24/01/06

18/01/06

17/01/06

11/01/06

10/01/06

09/01/06

07/01/06

05/01/06

30/12/05

23/12/05

22/12/05

21/12/05

05/12/05

01/12/05

29/11/05

28/11/05

13/10/05

05/10/05

04/10/05

23/09/05

21/09/05

#define tmp             r20     /* tmp val to work out key */
#define pnode           r19     /* pointer to node         */
#define node            r20     /* lower node word         */
#define key             r21     /* lookup key              */
#define multiplier      r22
#define size            r23     /* size of node level      */
#define cmp_value       r24     /* cmp val with guard      */
#define guard           r25     /* guard                   */
#define length          r26     /* guard length            */
#define type            r27     /* node type               */
#define shift           r28     /* justify guard shift     */
#define inc             r18     
                                
#define FIND_PTE(va, ppte, fail, ok, L1, L2)                    \
        rsm psr.dt;                                             \
        mov r19=IA64_KR(PT_BASE);                               \
        shr.u tmp = va,60;                                      \
        ;;                                                      \
        shr.u r17=r16,61;                                       \
        shl tmp = tmp,60;                                       \
        dep.z key = va,4,56;                                    \
        ;;                                                      \
        cmp.eq p6,p0=5,r17;                                     \
        add key = key, tmp;                                     \
        ;;                                                      \
        srlz.d;                                                 \
        LOAD_PHYSICAL(p6, r19, swapper_pg_dir);                 \
        shr.u key = key, (PAGE_SHIFT-10);         /* key set */ \
        ;;                                                      \
.L1:                                                            \
        ld8 node = [pnode];           /* get lower node word */ \
        ;;                                                      \
        extr.u type = node,0,2;                 /* node type */ \
        extr.u length = node,8,6;            /* guard length */ \
        extr.u guard = node,14,50;                  /* guard */ \
        extr.u size = node,2,6;                /* level size */ \
        ;;                                                      \
        sub shift = 64,length;                                  \
        sub multiplier = 64, size;                              \
        ;;                                                      \
        shl guard = guard,shift;  /* guard ready for compare */ \
        shr cmp_value = key,shift;                              \
        ;;                                                      \
        shl cmp_value = cmp_value, shift;   /* compare value */ \
        shl key = key,length;        /* Strip guard from key */ \
        ;;                                                      \
        shr.u inc = key,multiplier;                             \
        cmp.ne p6, p0 = 1,type;  /* If leaf or invalid break */ \
(p6)    br.cond.dptk .L2;                                       \
        ;;                                                      \
        cmp.ne p6, p0 = cmp_value,guard;                        \
(p6)    br.cond.dptk .L2;        /* If guards equal continue */ \
        add pnode = 8, pnode;          /* increment node ptr */ \
        shl key = key,size;           /* strip level from va */ \
        shl inc = inc,4;                                        \
        ;;                                                      \
        ld8 pnode = [pnode];    /* Get pointer to next level */ \
        ;;                                                      \
        cmp.eq p6,p7 = pnode,r0;                                \
(p6)    br.cond.dptk .L2;                  /* lookup failure */ \
        ;;                                                      \
        add pnode = pnode, inc;        /* point to next node */ \
        br .L1;                                      /* loop */ \
        ;;                                                      \
.L2:                                                            \
        cmp.ne p6, p7 = 2,type;                                 \
        add pnode = 8, pnode;                                   \
        ;;                                                      \
(p7)    cmp.ne p6, p7 = cmp_value,guard;                        \
        mov ppte = pnode;                                       \
        ;;                                                      \

20/09/05

19/09/05

Bad flu again. Hope to get back to work soon.

06/09/05

02/09/05

31/07/05

26/07/05

25/07/05

24/07/05

17/07/05

16/07/05

12/07/05

08/07/05

29/06/05

28/06/05

27/06/05

23/06/05

20/06/05

19/06/05

18/06/05

16/06/05

15/06/05

14/06/05

13/06/05

Assembler

#define node            r20     /* lower node word      */
#define key             r21     /* lookup key           */
#define pnode           r22     /* pointer to node      */
#define size            r23     /* size of node level   */
#define node_guard      r24     /* node guard           */
#define va_guard        r25     /* key guard            */
#define length          r26     /* node guard length    */
#define type            r27     /* node type            */
#define shift           r28     /* justify guard shift  */
#define multiplier va_guard     /* another shift        */
#define inc_node node_guard     /* point to right node  */
                                /* in level             */
#define retval          r29     /* return value         */
#define result          r8      

#define LOOKUP_GPT(ptr, addr)                                   \
        mov pnode = ptr;          /* point to root node      */ \
        mov key = addr;                                         \
        mov retval = r0;          /* default return NULL     */ \
        ;;                                                      \
        shl key = key, 14;        /* left justify 50 bit key */ \
.L1:                                                            \
        ld8 node = [pnode];       /* get lower node word     */ \
        ;;                                                      \
        extr.u type = node,0,2;   /* get node type           */ \
        extr.u length = node,8,6; /* length of node guard    */ \
        ;;                                                      \
        sub shift = 64,length;                                  \
        extr.u va_guard = node,14,50;                           \
        ;;                                                      \
        shl va_guard = va_guard,shift; /* va guard ready     */ \
        shr node_guard = key,shift;    /* for compare        */ \
        ;;                                                      \
        shl node_guard = node_guard, shift; /* node guard    */ \
        extr.u size = node,2,6;         /* ready for compare */ \
        ;;                                                      \
        cmp.eq p6, p0 = 2,type;             /* If leaf break */ \
(p6)    br.cond.dptk .L2;                                       \
        ;;                                                      \
        cmp.ne p6, p0 = node_guard,va_guard;                    \
(p6)    br.cond.dptk .L2;       /* If guards equal continue */  \
        ;;                                                      \
        shl key = key,length;       /* Strip guard from key */  \
        sub multiplier = 64, size;                              \
        add pnode = 8, pnode;   /* increment node ptr */        \
        ;;                                                      \
        shr.u inc_node = key,multiplier;                        \
        ld8 pnode = [pnode];    /* Get pointer to next level */ \
        ;;                                                      \
        cmp.eq p6,p0 = pnode,r0;                                \
(p6)    br.cond.dptk .L3;       /* Lookup failure - break    */ \
        ;;                                                      \
        shl key = key,size;     /* Strip level from VA       */ \
        shl inc_node = inc_node,4;                              \
        ;;                                                      \
        add pnode = pnode, inc_node;/* Point to next node    */ \
        br .L1;                     /* loop                  */ \
        ;;                                                      \
.L2:                                                            \
        cmp.ne p6, p0 = 2,type;                                 \
(p6)    br.cond.dptk .L3;       /* Not a Leaf - lookup fail  */ \
        ;;                                                      \
        cmp.ne p6, p0 = node_guard,va_guard;                    \
(p6)    br.cond.dptk .L3;       /* If guards OK - continue   */ \
        ;;                                                      \
        mov retval = pnode;     /* Return pointer to pte     */ \
        ;;                                                      \
        add retval = 8, retval;                                 \
        ;;                                                      \
.L3:    mov result = retval;                                    \
        ;;

08/06/05

07/06/05


.L1: /* Start for */

.L2:

.L3:


29/06/05

28/06/05

27/06/05

24/07/06

22/06/05

To test if the trie is holding the correct values. I will use the
iterator to output all trie values and compare with test rig's array.

21/06/05

20/06/05

18/06/05

17/06/05

16/06/05

15/06/05

14/06/05

02/01/05

01/01/05

18/05/05

18/05/05

12/05/05

1. Introduce general page table interface not including iterators. 2. Introduce all new general files required. 3. Rearrange all general code (pte_alloc_map etc). NOT CALLING interface yet. 4. Introduce iterators. 5. Call iterators. 6. Introduce page table interface specific to ia64. 7. Introduce files especially for ia64. 8. Rearrange files for ia64 fault.c init.c etc

12/05/05

11/05/05

5/05/05

28/04/05

27/04/05

25/04/05

22/04/05

21/04/05

15/04/05 - 20/04/05

14/04/05

12/04/05

12/04/05

11/04/05

10/04/05

09/04/05

08/04/05

07/04/05

05/04/05

01/04/05

31/03/05

17/03/05

16/03/05

16/03/05

16/03/05

15/03/05

15/03/05

10/02/05

6-9/02/05

5/02/05

4/0205

3/02/05

2/02/05

1/02/05

28/01/05

27/01/05

26/01/05

List of quick notes for myself.

25/01/05

21/01/05

20/01/05

19/01/05

18/01/05

17/01/05

14/01/05

12/01/05

11/01/05

10/01/05

09/01/05

08/01/05

07/01/05

06/01/05

05/01/05

04/01/05

30/12/04

To conform to Lucys interface I am planning to change:

From: void clear_page_tables(struct mmu_gather *tlb, unsigned long first, int nr)
To: void clear_page_tables(struct mm_struct *mm, unsigned long first, unsigned last)

From: int map_vm_area(struct vm_struct *area, pgprot_t prot, struct page ***pages)
To: int map_vm_area(struct vm_struct *area, pte_t pte, struct page ***pages)

From: static int map_area_pmd(pmd_t *pmd, unsigned long address, unsigned long size, pgprot_t prot, struct page ***pages)
To: static int map_area_pmd(pmd_t *pmd, unsigned long address, unsigned long size, pte_t pte, struct page ***pages)

From: change_protection(struct vm_area_struct *vma, unsigned long start, unsigned long end, pgprot_t newprot)
To: change_protection(struct vm_area_struct *vma, unsigned long start, unsigned long end, pte_t pte)

From: static void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long address, unsigned long end, struct zap_details *details) To: static void unmap_page_range(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long address, unsigned long end, struct zap_details *details) Still to look at struct zap_details.

more changes will follow ...

The following already conform to the interface:

void unmap_vm_area(struct vm_struct *area)

int copy_page_range(struct mm_struct *dst, struct mm_struct *src, struct vm_area_struct *vma)

struct page *follow_page(struct mm_struct *mm, unsigned long address, int write) Although int write doesn't conform with the interface it must stay, but it is implementation independent.

static inline int page_table_present(struct mm_struct *mm, unsigned long addr)

static inline int untouched_anonymous_page(struct mm_struct* mm, struct vm_area_struct *vma, unsigned long address)

The above are a quick scan through of part of the interface only. More to follow...

28/12/04

I am expecting to rework pte_lookup(struct mm_struct *mm, unsigned long address) which was added by Lucy Chubb for the new interface. Lucy intended for it to be rejigged as she commented it thus. I will be replacing the page table dependent code with a call to pte_lookup in follow_page.

27/12/04

For the last two days I have been trawling through the page table interface with a fine tooth comb. I have built myself a small model to enhance my understanding. I have also been closely examining Lucy's work on the PTI interface. Lucy has added several functions to the new interface already but they are yet to be used. I may incorporate them over the next few days - if I don't get side tracked.

23/12/04

22/12/04

17/12/04

Paul will be away from 20/12/04 until 03/01/05. I will be updating the wiki from time to time over the next couple of weeks.

16/12/04

15/12/04

14/12/04

13/12/04

10/12/04

9/12/04

I have decided to take both a top down and bottom up approach to the project. On the one hand I am reading all about the linux VM system which is extremely involved. I have decided to combine this with some experimental code to allow me to get a concrete understanding of the page table and to meet the interface. I want to develop a genuine feel for the page table and its interface and I think knocking up some experimental code is the way to do it.

8/12/04

Going out for an hour at 3:45PM.

7/12/04

Overview of Day

Trying to move away from the mechanics of working in the new environment to thinking about the project itself. Far more interesting.

6/12/04

Overview of Day

Kernel is up and running. Time to start the actual project...

3/12/04

2/12/04

Overview of Day

Debian successfully installed - unstable version. One major outstanding issue yet to be resoved. NIS is not working properly yet.

1/12/04

The network for my workstation is unavailable at the moment - so I am in drum learning to use the wiki and reading various documentation relevant to the project.

Overview of Day

My cse account was unavailable after around 3:30 while I waited for maintenace to be done.

30/11/04

Overview of Day

All in all a successful day. I now have reference point for the project ie. I can download, patch the kernel - and compile. I have yet to successfully boot the new kernel image to tartufi. This will go onto the top of my todo list. I still require much more time coming to terms with the environment. My C coding is much stronger than knowledge of the linux environment, however I aim to be comfortable (to my satisfaction) with my development environment by Monday.


CategoryTodo

IA64wiki: PaulDavies/ToDo (last edited 2009-12-10 03:13:42 by localhost)

Gelato@UNSW is sponsored by
the University of New South Wales National ICT Australia The Gelato Federation Hewlett-Packard Company Australian Research Council
Please contact us with any questions or comments.