ó
|
Y\¬Qc @ sÒ d Z d d l m Z d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
|
m Z d d l m Z m Z m Z d d l m Z d d l m Z m Z d d
|
l m Z d d l m Z e d Z d e f d YZ d Z d Z d Z d Z e j! d d f k r[d d l" Z" e" j# Z$ d Z% d Z& n e# Z$ e Z% e Z& d Z' d e f d YZ( d e) f d YZ* d e f d YZ+ d e* f d YZ, d S( sÛ Refactoring framework.
|
|
Used as a main program, this can refactor any number of files and/or
|
recursively descend down directories. Imported as a module, this
|
provides infrastructure to write your own refactoring tool.
|
iÿÿÿÿ( t with_statements# Guido van Rossum <guido@python.org>N( t chaini ( t drivert tokenizet token( t find_root( t pytreet pygram( t btm_utils( t btm_matcherc C s t | g g d g } t j j | j } g } xb t t j | D]K } | j d rI | j d rI | r | d } n | j | d qI qI W| S( sE Return a sorted list of all available fix names in the given package.t *t fix_s .pyi iýÿÿÿ(
|
t
|
__import__t ost patht dirnamet __file__t sortedt listdirt
|
startswitht endswitht append( t fixer_pkgt remove_prefixt pkgt fixer_dirt fix_namest name( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt get_all_fix_names! s t
|
_EveryNodec B s e Z RS( ( t __name__t
|
__module__( ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR . s c C sß t | t j t j f rC | j d k r3 t n t | j g St | t j rt | j rk t
|
| j St n t | t j rË t } x5 | j D]* } x! | D] } | j t
|
| q¦ Wq W| St d | d S( sf Accepts a pytree Pattern Node and returns a set
|
of the pattern types which will match first. s$ Oh no! I don't understand pattern %sN( t
|
isinstanceR t NodePatternt LeafPatternt typet NoneR t sett NegatedPatternt contentt _get_head_typest WildcardPatternt updatet Exception( t patt rt pt x( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR( 2 s c C s t j t } g } x£ | D] } | j r y t | j } Wn t k
|
r^ | j | q· XxU | D] } | | j | qf Wq | j d k rª | | j j | q | j | q Wx: t t
|
j j j t
|
j j D] } | | j | qÝ Wt | S( s^ Accepts a list of fixers and returns a dictionary
|
of head node type --> fixer list. N( t collectionst defaultdictt listt patternR( R R t _accept_typeR$ R R t python_grammart symbol2numbert
|
itervaluest tokenst extendt dict( t
|
fixer_listt
|
head_nodest everyt fixert headst node_type( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt _get_headnode_dictN s" c C s( g t | t D] } | d | ^ q S( sN
|
Return the fully qualified names for fixers in the package pkg_name.
|
t .( R t False( t pkg_namet fix_name( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt get_fixers_from_packageg s c C s | S( N( ( t obj( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt _identityn s i i c C s | j d d S( Nu
|
u
|
( t replace( t input( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt _from_system_newlinesu s c C s* t j d k r" | j d t j S| Sd S( Ns
|
u
|
( R t linesepRI ( RJ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt _to_system_newlinesw s c sà t } t j t j | j f d } t t j t j t j f } t
|
} ykxdt rÀ| \ } } | | k r q] q] | t j k r¦ | r Pn t } q] | t j k r¼| d k r¼| \ } } | t j k së | d k rï Pn | \ } } | t j k s| d k r Pn | \ } } | t j k rY| d k rY| \ } } n xa | t j k r¸| j | | \ } } | t j k s¢| d k r¦Pn | \ } } q\Wq] Pq] WWn t k
|
rÕn Xt | S( Nc s j } | d | d f S( Ni i ( t next( t tok( t gen( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt advance
s u fromu
|
__future__u importu (u ,( RC R t generate_tokenst StringIOt readlinet frozensetR t NEWLINEt NLt COMMENTR% t Truet STRINGt NAMEt OPt addt StopIteration( t sourcet have_docstringRQ t ignoret featurest tpt value( ( RP sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt _detect_future_features sD t
|
FixerErrorc B s e Z d Z RS( s A fixer could not be loaded.( R R t __doc__( ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRf ª s t RefactoringToolc B s! e Z i e d 6e d 6Z d Z d Z d d d Z d Z d Z d Z
|
d Z d Z e e d
|
Z e e d Z d Z e e d Z d Z e d Z d Z d Z d e d d Z d d Z d Z d Z d Z d Z d Z d Z d Z d Z RS( t print_functiont write_unchanged_filest FixR c C sÄ | | _ | p g | _ | j j | _ | d k rI | j j | n | j d re t j | _ n t j
|
| _ | j j d | _ g | _ t j d | _ g | _ t | _ t j | j d t j d | j | _ | j \ | _ | _ g | _ t j | _ g | _ g | _ x} t! | j | j D]f } | j" rT| j j# | q2| | j k rv| j j$ | q2| | j k r2| j j$ | q2q2Wt% | j | _& t% | j | _' d S( sÒ Initializer.
|
|
Args:
|
fixer_names: a list of fixers to import
|
options: an dict with configuration.
|
explicit: a list of fixers to run even if they are explicit.
|
Ri Rj Rh t convertt loggerN(( t fixerst explicitt _default_optionst copyt optionsR$ R* R t! python_grammar_no_print_statementt grammarR5 t getRj t errorst loggingt getLoggerRm t fixer_logRC t wroteR t DriverR Rl t
|
get_fixerst pre_ordert
|
post_ordert filest bmt BottomMatchert BMt bmi_pre_ordert bmi_post_orderR t BM_compatiblet add_fixerR RA t bmi_pre_order_headst bmi_post_order_heads( t selft fixer_namesRr Ro R> ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt __init__¶ s< c C sÝ g } g } x| j D]} t | i i d g } | j d d d } | j | j rr | t | j } n | j d } | j d j g | D] } | j ^ q } y t
|
| | } Wn' t k
|
rì t d | | f n X| | j | j }
|
|
|
j r?| j t k r?| | j k r?| j d | q n | j d | |
|
j d
|
k rn| j |
|
q |
|
j d k r| j |
|
q t d |
|
j q Wt j d } | j d | | j d | | | f S( s Inspects the options to load the requested patterns and handlers.
|
|
Returns:
|
(pre_order, post_order), where pre_order is the list of fixers that
|
want a pre-order AST traversal, and post_order is the list that want
|
post-order traversal.
|
R
|
RB i iÿÿÿÿt _t s Can't find %s.%ss Skipping implicit fixer: %ss Adding transformation: %st pret posts Illegal fixer order: %rt run_ordert key( Rn R t rsplitR t FILE_PREFIXt lent splitt CLASS_PREFIXt joint titlet getattrt AttributeErrorRf Rr Ry Ro RY t log_messaget log_debugt orderR t operatort
|
attrgettert sort( R t pre_order_fixerst post_order_fixerst fix_mod_patht modRE t partsR. t
|
class_namet fix_classR> t key_func( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR| ê s8 / c O s d S( s Called when an error occurs.N( ( R t msgt argst kwds( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt log_error s c G s' | r | | } n | j j | d S( s Hook to log a message.N( Rm t info( R R© Rª ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR s c G s' | r | | } n | j j | d S( N( Rm t debug( R R© Rª ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR s c C s d S( sT Called with the old version, new version, and filename of a
|
refactored file.N( ( R t old_textt new_textt filenamet equal( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt print_output! s c C sP xI | D]A } t j j | r5 | j | | | q | j | | | q Wd S( s) Refactor a list of files and directories.N( R R t isdirt refactor_dirt refactor_file( R t itemst writet doctests_onlyt dir_or_file( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt refactor& s c C sé t j d } xÕ t j | D]Ä \ } } } | j d | | j | j xe | D]] } | j d rW t j j | d | k rW t j j | | } | j | | | qW qW Wg | D] }
|
|
|
j d s¿ |
|
^ q¿ | (q Wd S( sÄ Descends down a directory and refactor every Python file found.
|
|
Python files are assumed to have a .py extension.
|
|
Files and subdirectories starting with '.' are skipped.
|
t pys Descending into %sRB i N(
|
R t extsept walkR R R R t splitextR R¶ ( R t dir_nameR¸ R¹ t py_extt dirpatht dirnamest filenamesR t fullnamet dn( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRµ / s
|
|
c C s y t | d } Wn' t k
|
r< } | j d | | d SXz t j | j d } Wd | j Xt | d d | } t | j
|
| f SWd QXd S( sG
|
Do our best to decode a Python source file correctly.
|
t rbs Can't open %s: %si NR- t encoding( NN( t opent IOErrorR¬ R$ R t detect_encodingRT t closet _open_with_encodingRK t read( R R± t ft errRÈ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt _read_python_sourceC s c C s | j | \ } } | d k r% d S| d 7} | r | j d | | j | | } | j sl | | k r | j | | | | | qþ | j d | nc | j | | } | j sÅ | rî | j rî | j t | d | d | d | n | j d | d S( s Refactors a file.Nu
|
s Refactoring doctests in %ss No doctest changes in %siÿÿÿÿR¸ RÈ s No changes in %s( RÑ R$ R t refactor_docstringRj t processed_filet refactor_stringt was_changedt unicode( R R± R¸ R¹ RJ RÈ t outputt tree( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR¶ S s
|
c C s· t | } d | k r* t j | j _ n zM y | j j | } Wn0 t k
|
ru } | j d | | j j | d SXWd | j | j _ X| | _
|
| j d | | j | | | S( sF Refactor a given input string.
|
|
Args:
|
data: a string holding the code to be refactored.
|
name: a human-readable name for use in error/log messages.
|
|
Returns:
|
An AST corresponding to the refactored input stream; None if
|
there were errors during the parse.
|
Ri s Can't parse %s: %s: %sNs Refactoring %s( Re R Rs R Rt t parse_stringR+ R¬ t __class__R t future_featuresR t refactor_tree( R t dataR Rb RØ RÐ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRÔ j s c C sÆ t j j } | ro | j d | j | d } | j sI | | k r_ | j | d | q | j d nS | j | d } | j s | rµ | j rµ | j t | d | n | j d d S( Ns Refactoring doctests in stdins <stdin>s No doctest changes in stdins No changes in stdin(
|
t syst stdinRÎ R RÒ Rj RÓ RÔ RÕ RÖ ( R R¹ RJ R× RØ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt refactor_stdin
s c
|
C s x- t | j | j D] } | j | | q W| j | j | j | j | j | j | j j | j } xçt
|
| j rcxÎ| j j D]À} | | k r | | r | | j d t j j d t | j rþ | | j d t j j n x[t | | D]F} | | | k r9| | j | n y t | Wn t k
|
r]qn X| j r|| | j k r|qn | j | } | r| j | | } | d k rU| j | x9 | j D]+ } | j säg | _ n | j j | qÉW| j j | j } x? | D]4 } | | k r6g | | <n | | j | | qWqUqqWq q Wq} Wx- t | j | j D] } | j | | qzW| j S( sÏ Refactors a parse tree (modifying the tree in place).
|
|
For compatible patterns the bottom matcher module is
|
used. Otherwise the tree is traversed node-to-node for
|
matches.
|
|
Args:
|
tree: a pytree.Node instance representing the root of the tree
|
to be refactored.
|
name: a human-readable name for this tree.
|
|
Returns:
|
True if the tree was modified, False otherwise.
|
R t reverseN(! R R} R~ t
|
start_treet traverse_byR R R t runt leavest anyt valuesRn R R t Baset depthRY t keep_line_ordert
|
get_linenoR2 t removeR t
|
ValueErrort fixers_appliedt matcht transformR$ RI R R9 t finish_treeRÕ (
|
R RØ R R> t match_sett nodet resultst newt new_matchest fxr( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRÜ sJ . c C s | s
|
d Sxr | D]j } xa | | j D]R } | j | } | r% | j | | } | d k rw | j | | } qw q% q% Wq Wd S( s Traverse an AST, applying a set of fixers to each node.
|
|
This is a helper method for refactor_tree().
|
|
Args:
|
fixers: a list of fixer instances.
|
traversal: a generator that yields AST nodes.
|
|
Returns:
|
None
|
N( R# Rï Rð R$ RI ( R Rn t traversalRó R> Rô Rõ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRã ä s c C s½ | j j | | d k rB | j | d } | d k rB d Sn | | k } | j | | | | | r | j d | | j s d Sn | r© | j | | | | n | j d | d S( sR
|
Called when a file has been refactored and there may be changes.
|
i Ns No changes to %ss Not writing changes to %s( R R R$ RÑ R³ R Rj t
|
write_file( R R° R± R¯ R¸ RÈ R² ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRÓ û s c C s¸ y t | d d | } Wn* t j k
|
rE } | j d | | d SXzG y | j t | Wn) t j k
|
r } | j d | | n XWd | j X| j d | t | _ d S( sÑ Writes a string to a file.
|
|
It first shows a unified diff between the old text and the new text, and
|
then rewrites the file; the latter is only done if the write option is
|
set.
|
t wRÈ s Can't create %s: %sNs Can't write %s: %ss Wrote changes to %s(
|
RÍ R t errorR¬ R¸ RM RÌ R RY Rz ( R R° R± R¯ RÈ RÏ RÐ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRù s s >>> s ... c
|
C s g } d } d } d } d } x+| j t D]} | d 7} | j j | j r² | d k r | j | j | | | | n | } | g } | j | j } | | } q. | d k r| j | | j sñ | | | j j
|
d k r| j | q. | d k r/| j | j | | | | n d } d } | j | q. W| d k rz| j | j | | | | n d j | S( sË Refactors a docstring, looking for doctests.
|
|
This returns a modified version of the input string. It looks
|
for doctests, which start with a ">>>" prompt, and may be
|
continued with "..." prompts, as long as the "..." is indented
|
the same as the ">>>".
|
|
(Unfortunately we can't use the doctest module's parser,
|
since, like most parsers, it is not geared towards preserving
|
the original source.)
|
i i u
|
u N( R$ t
|
splitlinesRY t lstripR t PS1R9 t refactor_doctestt findt PS2t rstripR R (
|
R RJ R± t resultt blockt block_linenot indentt linenot linet i( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRÒ ( s:
|
c
|
C ss y | j | | | } Wnu t k
|
r } | j j t j rm x* | D] } | j d | j d qG Wn | j d | | | j j
|
| | SX| j | | rot | j t } | | d | | d } } | d g | d k sø t | | d j d s | d c d 7<n | | j | j d g } | ro| g | D] } | | j | ^ qK7} qon | S( sÞ Refactors one doctest.
|
|
A doctest is given as a block of lines, the first of which starts
|
with ">>>" (possibly indented), while the remaining lines start
|
with "..." (identically indented).
|
|
s
|
Source: %su
|
s+ Can't parse docstring in %s line %s: %s: %si iÿÿÿÿi ( t parse_blockR+ Rm t isEnabledForRw t DEBUGR R R¬ RÚ R RÜ RÖ Rü RY t AssertionErrorR Rþ t popR (
|
R R R R R± RØ RÐ R Rõ t clipped( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRÿ S s& # .c C s | j r d } n d } | j s4 | j d | n1 | j d | x | j D] } | j | qN W| j r | j d x! | j D] } | j | q
Wn | j rt | j d k rÍ | j d n | j d t | j x0 | j D]" \ } } } | j | | | qð Wn d S( Nt weres
|
need to bes No files %s modified.s Files that %s modified:s$ Warnings/messages while refactoring:i s There was 1 error:s There were %d errors:( Rz R R Ry Rv R ( R R t filet messageR© Rª R« ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt summarizep s$ c C s1 | j j | j | | | } t | _ | S( s³ Parses a block into a tree.
|
|
This is necessary to get correct line number / offset information
|
in the parser diagnostics and embedded into the parse tree.
|
( R t parse_tokenst wrap_toksRU RÛ ( R R R R RØ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR
|
s ! c c s t j | j | | j } xe | D]] \ } } \ } } \ } }
|
} | | d 7} | | d 7} | | | | f | |
|
f | f Vq% Wd S( s; Wraps a tokenize stream to systematically modify start/end.i N( R RR t gen_linesRN ( R R R R R8 R# Rd t line0t col0t line1t col1t line_text( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR s
|
(c c s¢ | | j } | | j } | } xi | D]a } | j | rN | t | Vn4 | | j d k rl d Vn t d | | f | } q' Wx t r d Vq Wd S( s Generates lines as expected by tokenize from a list of lines.
|
|
This strips the first len(indent + self.PS1) characters off each line.
|
u
|
s line=%r, prefix=%rR N( Rþ R R R R R RY ( R R R t prefix1t prefix2t prefixR ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR s
|
N( R R RC Rp R R R$ R R| R¬ R R R³ R» Rµ RÑ R¶ RÔ Rà RÜ Rã RÓ Rù Rþ R RÒ Rÿ R R
|
R R ( ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyRh ® s:
|
|
4 ( O +
|
t MultiprocessingUnsupportedc B s e Z RS( ( R R ( ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR ³ s t MultiprocessRefactoringToolc B s5 e Z d Z e e d d Z d Z d Z RS( c O s/ t t | j | | d | _ d | _ d S( N( t superR R R$ t queuet output_lock( R Rª t kwargs( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR ¹ s i c C sf | d k r( t t | j | | | Sy d d l } Wn t k
|
rQ t n X| j d k rp t d n | j | _ | j
|
| _ g t | D] } | j d | j ^ q } z; x | D] } | j qÆ Wt t | j | | | Wd | j j x$ t | D] } | j j d qWx' | D] } | j r5| j q5q5Wd | _ Xd S( Ni iÿÿÿÿs already doing multiple processest target( R! R R» t multiprocessingt ImportErrorR R" R$ t RuntimeErrort JoinableQueuet LockR# t xranget Processt _childt startR t putt is_alive( R R· R¸ R¹ t num_processesR& R t processesR. ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR» ¾ s2
|
+ c C so | j j } xY | d k rj | \ } } z t t | j | | Wd | j j X| j j } q Wd S( N( R" Ru R$ R! R R¶ t task_done( R t taskRª R$ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR- Û s c O sE | j d k r( | j j | | f n t t | j | | Sd S( N( R" R$ R/ R! R R¶ ( R Rª R$ ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR¶ æ s ( R R R RC R» R- R¶ ( ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyR · s
|
(- Rg t
|
__future__R t
|
__author__R RÞ Rw R R0 RS t itertoolsR t pgen2R R R t
|
fixer_utilR R R R R t buR R RY R R+ R R( RA RF RH t version_infot codecsRÉ RÍ RK RM Re Rf t objectRh R R ( ( ( sX /tmp/ndk-User/buildhost/install/prebuilt/darwin-x86_64/lib/python2.7/lib2to3/refactor.pyt <module> sF (ÿ ÿ
|
