o iq@sUdZddlmZddlZddlZddlZddlZddlZddlZddl m Z m Z m Z ddl mZddlmZmZmZmZmZmZmZeejjZedZedkZd ed <eed d d d Z e!e Z"Wdn1srwYded<ded<ded<ded<eedd d dZ e!e Z#e#dZ$e#dZ%e#dZ&e#dZ'Wdn1swYdZ(dZ)e*dde&+DZ,dmd!d"Z-dmd#d$Z.dnd'd(Z/dod+d,Z0dpd.d/Z1dqd3d4Z2drd7d8Z3dsd;d<Z4dtdBdCZ5GdDdEdEej6Z7GdFdGdGZ8e8j9e8j:e8j;e8je8j?ge8j@gdHZAe8j9ge8j:e8j;e8je8j?geAdIdHe8j9e8j:ge8j;e8je8j?geAdIdHeAeAeAeAdJZBe GdKdLdLe ZCdudOdPZDdvdSdTZEdvdUdVZFdvdWdXZGdvdYdZZHdvd[d\ZIdwd`daZJdwdbdcZKdxdkdlZLdS)yzBasic tests for Parts I & II) annotationsN)Flagautounique)Path)AnyCallableDictListOptionalSequenceTypetestsdarwinzdict[str, Any]EXPECTED_RESULTSzexpected_results.jsonrzutf-8)encodingzdict[str, List[str]]EXTRA_CREDIT_PROGRAMS List[str]REQUIRES_MATHLIBzDict[str, List[str]] DEPENDENCIESASSEMBLY_DEPENDENCIESztest_properties.jsonextra_credit_testsrequires_mathlib assembly_libslibsz_osx.sz_linux.sccs:|]}|D]}|t|tfD]}t|jVqqqdSN) MAC_SUFFIX LINUX_SUFFIXrname).0rlibplatform_specific_libr#`/home/gtoal/gtoal.com/languages/algol60/new-unicode-parser/lang/c/tests4/test_framework/basic.py )sr%returnstrcCs trdSdS)Nos_xlinux)IS_OSXr#r#r#r$ get_platform3r+cCs trtStSr)r*rrr#r#r#r$get_platform_suffix:r,r- source_filercCs4|jdrt||jdtd }t|tS)zkey to use in EXPECTED_RESULTS, REQUIRES_MATHLIB, EXTRA_CREDIT_PROGRAMS If this ends with _client.c, use corresponding lib as props key _clientN)stemendswith replace_stemlenr' relative_toTEST_DIR)r.r#r#r$ get_props_keyAs r6progboolcCst|}|tvo t Sr)r6rr*)r7keyr#r#r$ needs_mathlibJsr: List[Path]c Csnt|}g}|tvrt|D]}|t}t|}||q|tvr5t|}|D] }t|}||q)|S)zNGet extra libraries this test program depends on (aside from lib/client pairs))r6rr-r5appendr) r7 props_keyrasm_dep asm_filenameasm_path lib_filenamesllib_pathr#r#r$get_libsOs    rDproc subprocess.CompletedProcess[str]NonecCs|jr t|jdSdS)zcPrint out stderr of CompletedProcess if it's not empty. Intended to print assembler/linker warningsN)stderrprint)rEr#r#r$ print_stderr`srJ source_filesoptionsc Cs~|dd}ztjgd||d|gdddd}t|Wntjy3}zt|j|d}~wwtj|gdddd d S) a7Compile input files using 'gcc' command and run the resulting executable Args: source_files: list of input files - could be C, assembly, or object files options: command-line options Returns: a CompletedProecess object that captures the executable's return code and output r)gccz-DSUPPRESS_WARNINGSz-oT)checktextcapture_outputNF$@)rPrQrRtimeout) with_suffix subprocessrunrJCalledProcessError RuntimeErrorrH)rKrLexeresulterrr#r#r$gcc_compile_and_runfs    r]pathnew_stemcCs&tjdkr ||S|||jS)z;Return a new path with the stem changed and suffix the same) )sys version_info with_stem with_namerUsuffix)r^r_r#r#r$r2s  r2expected_retcodeintexpected_stdoutactualexe_namecCsd|g}||jkr|d|d|jg7}|j|kr>|dt|dt|jtt||j}|||j rL|d||j d |S)aBuild the error message for when a compiled test program behaves incorrectly Called when a unittest assert* message fails Args: expected_retcode: expected return code from EXPECTED_RESULTS expected_stdout: expected stdout from EXPECTED_RESULTS (often empty) actual: result from calling subprocess.run() on compiled test program exe_name: full path to compiled test program Returns: an error message zIncorrect behavior in z* Bad return code: expected z and got z* Bad stdout: expected z+* Expected no output to stderr, but found:  ) returncodestdoutr<reprlistdifflibndiff splitlinesextendrHjoin)rgrirjrk msg_linesdiffr#r#r$build_error_messages"       rxc@seZdZUdZdZded<ded<ded<ded <d ed <d-ddZ d.d/ddZd0ddZd1ddZ d0ddZ d0ddZ d0d d!Z d2d%d&Z d3d(d)Zd4d+d,ZdS)5 TestChaptera]Base per-chapter test class - should be subclassed, not instantiated directly. For each chapter under test, we construct a subclass of TestChapter and generate a test method for each C program in the corresponding directory. Each dynamically generated test calls one of the main test methods defined below: * compile_failure: compilation should fail) * compile_success: compilation should succeed up to some intermediate stage) * compile_and_run: compiling and running the test program should give the expected result) * compile_client_and_run: the test program consists of a client and library. compiling the client with our compiler and library with the system compiler, run the compiled program, and validate the result * compile_lib_and_run: like compile_client_and_run, but compile the *library* withour compiler and *client* with the system compiler * compile_with_helper_lib_and_run: like compile_client_and_run except the library is defined in test_properties.json and is not under test library should be in TEST_DIR/helper_libs/ * compile_with_asm_lib_and_run: like compile_client_and_run except the library is an assembly file defined in test_properties.json, not a C file The other methods in TestChapter are all utilties called by the compile_* methods. Frtest_dirccz list[str]rLr' exit_stage list[int] error_codesr&rGcCs,dd|jdD}|D]}|q dS)zKDelete files produced during this test run (e.g. assembly and object files)css0|]}|s|jdvr|jtvr|VqdS)).cz.hz.mdN)is_dirrfr ASSEMBLY_LIBSr fr#r#r$r%s  z'TestChapter.tearDown..*N)rzrglobunlink)self garbage_filesjunkr#r#r$tearDowns   zTestChapter.tearDownNr.cc_opt Optional[str]rFcCs^|dur|jdurd|j}|jg|j}|dur||||tj|dddd}|S)aCompile the test program (possibly up to some intermediate stage), but don't run it. Args: source_file: Absolute path to source file cc_opt (optional): Additional command-line options to pass to compiler (in addition to exit stage and anything specified in self.options). Used to compile without linking (for library tests); to link math library; and to compile to assembly (for optimization tests) Returns: A CompletedObject the captures the result of compilation (including an exit code indicating whether it succeeded and any error messages produced by the compiler) Nz--TF)rRrPrQ)r|r{rLr<rVrW)rr.rargsrEr#r#r$invoke_compilers   zTestChapter.invoke_compilercCsL|j}|j|d}|j|d|dd|j|}||dS)zMake sure the compiler under test didn't emit executable or assembly code. Used when compiling invalid test cases or testing intermediate stages.z.szFound assembly file z for invalid program!msgN)r0parent assertFalseexists)rr.r0 assembly_fileexecutable_filer#r#r$validate_no_outputs  zTestChapter.validate_no_outputrjcCst|}t|}|d}|dd}|jd}|j||jt||||d|j||jt||||d|j|j t||||ddS)aHValidate that the running compiled executable gave the expected result. Compare return code and stdout to values in EXPECTED_RESULTS. Args: source_file: Absolute path of the source file for a test program actual: result of compiling this source file with self.cc and running it return_codernrMrrN) r6rgetr assertEqualrmrxrnrrH)rr.rjr9expectedrgrirZr#r#r$ validate_runs(s&       zTestChapter.validate_runscCs|jtjd|d||}|Wdn1swY|jrDddd|jD}|j|j|jd|jd|d d| |dS) aTest that compiling an invalid program returns a non-zero exit code Use this when compilation of the test program should fail at or before the stage under test. E.g. if type_error.c contains a type error, when we use the --stage validate option, test_type_error will call compile_failure but when we use the --stage parse option, test_type_error will call compile_success (below) zDidn't catch error in rNz, cs|]}t|VqdSr)r')r er#r#r$r%[z.TestChapter.compile_failure..z"Compilation failed with exit code z0, but expected one of the following exit codes: .) assertRaisesrVrXrcheck_returncoder~ruassertInrmr)rr.r[exit_code_listr#r#r$compile_failureJs   zTestChapter.compile_failurecCs8||}|j|jdd|d|jd||dS)aTest that compiling a valid program returns exit code of 0. Use this when compilation of the program should succeed up until the stage under test. This is only used when testing an intermediate stage; when testing the whole compiler, use compile_and_run instead.rcompilation of  failed with error: rN)rrrmrHr)rr.r[r#r#r$compile_successds zTestChapter.compile_successcCst|}|r|||dSt|rd}nd}|j||d}|j|jdd|d|jdt||d}t j |gd d d d d }| ||dS) z>Compile a valid test program, run it, and validate the resultsN-lmrrrrrrMFTrS)rPrRrQrT) rDlibrary_test_helperr:rrrmrHrJrUrVrWr)rr. extra_libsrcompile_resultrZr[r#r#r$compile_and_runus&   zTestChapter.compile_and_runfile_under_test other_filesr;c Cs|jdkr(|j|dd}|j|jdd|d|jdt||d}|}n|}|d}|g|}g}t|sCtd d |DrH| d t ||}| ||d S) a}Compile one file in a multi-file program and validate the results. Compile file_under_test with compiler under test and other_files with 'gcc' command. Link 'em together, run the resulting executable, make validate the results. Args: file_under_test: Absolute path of one file in a multi-file program. Usually a C file we want to compile with self.cc, but sometimes (in optimization tests) an assembly file that we've alerady compiled with self.cc and inspected other_files: Absolute paths to other files in the multi-file program rz-crrrrrz.ocsrr)r:rr#r#r$r%rz2TestChapter.library_test_helper..rN) rfrrrmrHrJrUr:anyr<r]r) rrrcompilation_resultcompiled_file_under_testvalidation_keyrKrLr[r#r#r$rs$      zTestChapter.library_test_helper client_pathcCs,t||jdtd }|||gdS)z>Multi-file program test where our compiler compiles the clientNr/)r2r0r3r)rrrCr#r#r$compile_client_and_runsz"TestChapter.compile_client_and_runrCcCs"t||jd}|||gdS)z?Multi-file program test where our compiler compiles the libraryr/N)r2r0r)rrCrr#r#r$compile_lib_and_runszTestChapter.compile_lib_and_run)r&rGr)r.rrrr&rF)r.rr&rG)r.rrjrFr&rG)rrrr;r&rG)rrr&rG)rCrr&rG)__name__ __module__ __qualname____doc__ longMessage__annotations__rrrrrrrrrrr#r#r#r$rys&     "   $ .ryc@s0eZdZdZdZdZdZdZdZdZ dZ d Z d S) TestDirsz-Subdirectory names within each test directory invalid_lex invalid_parseinvalid_semanticsinvalid_declarations invalid_typesinvalid_labelsinvalid_struct_tagsvalidN) rrrr INVALID_LEX INVALID_PARSEINVALID_SEMANTICSINVALID_DECLARATIONS INVALID_TYPESINVALID_LABELSINVALID_STRUCT_TAGSVALIDr#r#r#r$rsr)invalidrr)lexparsevalidatetackycodegenrWc@sZeZdZdZeZeZeZeZeZ eZ eZ dZ eeBeBeBe Be Be BZ dS) ExtraCreditzEAn ExtraCredit flag represents a set of enabled extra-credit featuresrN)rrrrrBITWISECOMPOUND INCREMENTGOTOSWITCHNANUNIONNONEALLr#r#r#r$rs r source_progextra_credit_flagscs>d|jvrdSt|}ddt|D}tfdd|DS)a^Based on our current extra credit settings, should we include this test program? Args: source_prog: Absolute path to a C test program extra_credit_flags: extra credit features to test (specified on the command line) Returns: true if we should _exclude_ this program from test run, false if we should include it. extra_creditFcss|] }tt|VqdSr)rr'upper)r featurer#r#r$r%<s z(excluded_extra_credit..c3s|]}|vVqdSrr#rrr#r$r%Br)partsr6rr)rrr9features_requiredr#rr$excluded_extra_credit)s rprogramCallable[[TestChapter], None]cdfdd }|S) z4Generate a test method for an invalid source programrryr&rGc|dSr)rrrr#r$ test_invalidHz'make_invalid_test..test_invalidNrryr&rGr#)rrr#rr$make_invalid_testErcr) zGenerate one test method to compile a valid program. Only used when testing intermediate stages. Use make_test_run when testing the whole compilerrryr&rGcrr)rrrr#r$ test_validTrz#make_test_valid..test_validNrr#)rrr#rr$make_test_validNsrcr) zGenerate one test method to compile and run a valid single-file program (the program may depend on additional source or assembly files that are not under test) rryr&rGcrr)rrrr#r$test_run_rzmake_test_run..test_runNrr#)rrr#rr$ make_test_runZsrcr) z9Generate one test method for client in multi-file programrryr&rGcrr)rrrr#r$ test_clienthrz%make_test_client..test_clientNrr#)rrr#rr$make_test_clienterrcr) z:Generate one test method for library in multi-file programrryr&rGcrr)rrrr#r$test_libqrzmake_test_lib..test_libNrr#)rrr#rr$ make_test_libnrrrzstage/list[tuple[str, Callable[[TestChapter], None]]]c Csng}t|dD],}||}|dD] }t||rq||d}d|}t|} ||| fqq|S)aGenerate one test method for each invalid test program in test_dir. We use extra_credit_flags and stage to discover invalid test cases within test_dir. Args: test_dir: Absolute path to the test directory for a specific chapter (e.g. /path/to/write-a-c-compiler-tests/chapter_1/) stage: only compile programs through this stage. this dictates which programs are considered invalid (e.g. if stage is "parse" programs with type errors are valid, because we stop before typechecking) extra_credit_flags: extra credit features to test (specified on the command line) Returns: A list of (name, test method) tuples, intended to be included on a dynamically generated subclass of TestChapter r*.crMtest_)DIRECTORIES_BY_STAGErrr4rUrr<) rzrrrinvalid_subdirinvalid_test_dirrr9 test_name test_methodr#r#r$make_invalid_testsws   rc Csg}t|dD]J}||}|dD]>}t||rq||d}d|}|dkrFd|jvr6t|} n|jdrAt |} n t |} nt |} | || fqq|S)aGenerate one test method for each valid test program in test_dir. We use stage and extra_credit_flags to discover valid test cases in test_dir. We also use stage to determine what sort of test to run (e.g. if stage is "run" we actually run the executable we compile; otherwise we just check that compilation succeeded). Args: test_dir: Absolute path to the test directory for a specific chapter (e.g. /path/to/write-a-c-compiler-tests/chapter_1/) stage: only compile programs through this stage. this dictates which programs are considered valid (e.g. if stage is "parse" programs with type errors are valid, because we stop before typechecking) extra_credit_flags: extra credit features to test (specified on the command line) Returns: A list of (name, test method) tuples, intended to be included on a dynamically generated subclass of TestChapter rrrMrrW librariesclient) rrrr4rUrrr0r1rrrr<) rzrrr valid_subdir valid_testdirrr9rrr#r#r$make_valid_testss$       rcompilerchapter Sequence[str] skip_invalidr~r}Type[unittest.TestCase]cCstd|}d|}||||dkrdn||d} |s0t|||} | D]\} } | | | <q't|||} | D]\} } | | | <q8t|tf| S)aConstruct the test class for a normal (non-optimization) chapter. Construct a subclass of TestChapter, generating a test method for each program in this chapter's test suite (possibly including some extra credit programs, depending on the extra_credit argument). Args: compiler: absolute path to compiler under test chapter: the chapter we're testing options: extra command-line options to pass through to compiler stage: only compile programs up through this stage extra_credit_flags: extra credit features to test, represented as a bit vector skip_invalid: true if we should skip invalid test programs error_codes: expected compiler exit codes when rejecting invalid programs chapter_ryrWN)rzr{rLr|r~)r5joinpathresolverrtypery)r r rLrrr r~rztestclass_nametestclass_attrs invalid_testsrtest_cls valid_testsr#r#r$build_test_classs        r)r&r')r.rr&r')r7rr&r8)r7rr&r;)rErFr&rG)rKr;rLrr&rF)r^rr_r'r&r) rgrhrir'rjrFrkr'r&r')rrrrr&r8)rrr&r)rzrrr'rrr&r)r rr rhrLr rr'rrr r8r~r}r&r )Mr __future__rrqjsonplatformrVrbunittestenumrrrpathlibrtypingrrr r r r r __file__rROOT_DIRr5systemlowerr*ropenrloadr test_inforrrrrrsetvaluesrr+r-r6r:rDrJr]r2rxTestCaseryrrrrrrrrrdirsrrrrrrrrrrrr#r#r#r$s  $          % +      &3