| .. | .. |
|---|
| 35 | 35 | program order (po) relation (on the same CPU). It guarantees that |
|---|
| 36 | 36 | each ``atomic_*()`` and ``refcount_*()`` operation is atomic and instructions |
|---|
| 37 | 37 | are executed in program order on a single CPU. |
|---|
| 38 | | -This is implemented using :c:func:`READ_ONCE`/:c:func:`WRITE_ONCE` and |
|---|
| 38 | +This is implemented using READ_ONCE()/WRITE_ONCE() and |
|---|
| 39 | 39 | compare-and-swap primitives. |
|---|
| 40 | 40 | |
|---|
| 41 | 41 | A strong (full) memory ordering guarantees that all prior loads and |
|---|
| .. | .. |
|---|
| 44 | 44 | It also guarantees that all po-earlier stores on the same CPU |
|---|
| 45 | 45 | and all propagated stores from other CPUs must propagate to all |
|---|
| 46 | 46 | other CPUs before any po-later instruction is executed on the original |
|---|
| 47 | | -CPU (A-cumulative property). This is implemented using :c:func:`smp_mb`. |
|---|
| 47 | +CPU (A-cumulative property). This is implemented using smp_mb(). |
|---|
| 48 | 48 | |
|---|
| 49 | 49 | A RELEASE memory ordering guarantees that all prior loads and |
|---|
| 50 | 50 | stores (all po-earlier instructions) on the same CPU are completed |
|---|
| .. | .. |
|---|
| 52 | 52 | stores on the same CPU and all propagated stores from other CPUs |
|---|
| 53 | 53 | must propagate to all other CPUs before the release operation |
|---|
| 54 | 54 | (A-cumulative property). This is implemented using |
|---|
| 55 | | -:c:func:`smp_store_release`. |
|---|
| 55 | +smp_store_release(). |
|---|
| 56 | + |
|---|
| 57 | +An ACQUIRE memory ordering guarantees that all post loads and |
|---|
| 58 | +stores (all po-later instructions) on the same CPU are |
|---|
| 59 | +completed after the acquire operation. It also guarantees that all |
|---|
| 60 | +po-later stores on the same CPU must propagate to all other CPUs |
|---|
| 61 | +after the acquire operation executes. This is implemented using |
|---|
| 62 | +smp_acquire__after_ctrl_dep(). |
|---|
| 56 | 63 | |
|---|
| 57 | 64 | A control dependency (on success) for refcounters guarantees that |
|---|
| 58 | 65 | if a reference for an object was successfully obtained (reference |
|---|
| .. | .. |
|---|
| 71 | 78 | |
|---|
| 72 | 79 | Function changes: |
|---|
| 73 | 80 | |
|---|
| 74 | | - * :c:func:`atomic_set` --> :c:func:`refcount_set` |
|---|
| 75 | | - * :c:func:`atomic_read` --> :c:func:`refcount_read` |
|---|
| 81 | + * atomic_set() --> refcount_set() |
|---|
| 82 | + * atomic_read() --> refcount_read() |
|---|
| 76 | 83 | |
|---|
| 77 | 84 | Memory ordering guarantee changes: |
|---|
| 78 | 85 | |
|---|
| .. | .. |
|---|
| 84 | 91 | |
|---|
| 85 | 92 | Function changes: |
|---|
| 86 | 93 | |
|---|
| 87 | | - * :c:func:`atomic_inc` --> :c:func:`refcount_inc` |
|---|
| 88 | | - * :c:func:`atomic_add` --> :c:func:`refcount_add` |
|---|
| 94 | + * atomic_inc() --> refcount_inc() |
|---|
| 95 | + * atomic_add() --> refcount_add() |
|---|
| 89 | 96 | |
|---|
| 90 | 97 | Memory ordering guarantee changes: |
|---|
| 91 | 98 | |
|---|
| .. | .. |
|---|
| 96 | 103 | |
|---|
| 97 | 104 | Function changes: |
|---|
| 98 | 105 | |
|---|
| 99 | | - * :c:func:`atomic_dec` --> :c:func:`refcount_dec` |
|---|
| 106 | + * atomic_dec() --> refcount_dec() |
|---|
| 100 | 107 | |
|---|
| 101 | 108 | Memory ordering guarantee changes: |
|---|
| 102 | 109 | |
|---|
| .. | .. |
|---|
| 108 | 115 | |
|---|
| 109 | 116 | Function changes: |
|---|
| 110 | 117 | |
|---|
| 111 | | - * :c:func:`atomic_inc_not_zero` --> :c:func:`refcount_inc_not_zero` |
|---|
| 112 | | - * no atomic counterpart --> :c:func:`refcount_add_not_zero` |
|---|
| 118 | + * atomic_inc_not_zero() --> refcount_inc_not_zero() |
|---|
| 119 | + * no atomic counterpart --> refcount_add_not_zero() |
|---|
| 113 | 120 | |
|---|
| 114 | 121 | Memory ordering guarantees changes: |
|---|
| 115 | 122 | |
|---|
| .. | .. |
|---|
| 119 | 126 | result of obtaining pointer to the object! |
|---|
| 120 | 127 | |
|---|
| 121 | 128 | |
|---|
| 122 | | -case 5) - decrement-based RMW ops that return a value |
|---|
| 123 | | ------------------------------------------------------ |
|---|
| 129 | +case 5) - generic dec/sub decrement-based RMW ops that return a value |
|---|
| 130 | +--------------------------------------------------------------------- |
|---|
| 124 | 131 | |
|---|
| 125 | 132 | Function changes: |
|---|
| 126 | 133 | |
|---|
| 127 | | - * :c:func:`atomic_dec_and_test` --> :c:func:`refcount_dec_and_test` |
|---|
| 128 | | - * :c:func:`atomic_sub_and_test` --> :c:func:`refcount_sub_and_test` |
|---|
| 129 | | - * no atomic counterpart --> :c:func:`refcount_dec_if_one` |
|---|
| 134 | + * atomic_dec_and_test() --> refcount_dec_and_test() |
|---|
| 135 | + * atomic_sub_and_test() --> refcount_sub_and_test() |
|---|
| 136 | + |
|---|
| 137 | +Memory ordering guarantees changes: |
|---|
| 138 | + |
|---|
| 139 | + * fully ordered --> RELEASE ordering + ACQUIRE ordering on success |
|---|
| 140 | + |
|---|
| 141 | + |
|---|
| 142 | +case 6) other decrement-based RMW ops that return a value |
|---|
| 143 | +--------------------------------------------------------- |
|---|
| 144 | + |
|---|
| 145 | +Function changes: |
|---|
| 146 | + |
|---|
| 147 | + * no atomic counterpart --> refcount_dec_if_one() |
|---|
| 130 | 148 | * ``atomic_add_unless(&var, -1, 1)`` --> ``refcount_dec_not_one(&var)`` |
|---|
| 131 | 149 | |
|---|
| 132 | 150 | Memory ordering guarantees changes: |
|---|
| 133 | 151 | |
|---|
| 134 | 152 | * fully ordered --> RELEASE ordering + control dependency |
|---|
| 135 | 153 | |
|---|
| 136 | | -.. note:: :c:func:`atomic_add_unless` only provides full order on success. |
|---|
| 154 | +.. note:: atomic_add_unless() only provides full order on success. |
|---|
| 137 | 155 | |
|---|
| 138 | 156 | |
|---|
| 139 | | -case 6) - lock-based RMW |
|---|
| 157 | +case 7) - lock-based RMW |
|---|
| 140 | 158 | ------------------------ |
|---|
| 141 | 159 | |
|---|
| 142 | 160 | Function changes: |
|---|
| 143 | 161 | |
|---|
| 144 | | - * :c:func:`atomic_dec_and_lock` --> :c:func:`refcount_dec_and_lock` |
|---|
| 145 | | - * :c:func:`atomic_dec_and_mutex_lock` --> :c:func:`refcount_dec_and_mutex_lock` |
|---|
| 162 | + * atomic_dec_and_lock() --> refcount_dec_and_lock() |
|---|
| 163 | + * atomic_dec_and_mutex_lock() --> refcount_dec_and_mutex_lock() |
|---|
| 146 | 164 | |
|---|
| 147 | 165 | Memory ordering guarantees changes: |
|---|
| 148 | 166 | |
|---|
| 149 | 167 | * fully ordered --> RELEASE ordering + control dependency + hold |
|---|
| 150 | | - :c:func:`spin_lock` on success |
|---|
| 168 | + spin_lock() on success |
|---|