$title Implicit Data Contract Test (IDC01,SEQ=807) $onText Contributor: M. Bussieck, December 2019 $offText * * GAMS Program with external input and output with data statements * Can be configured to have uels for input symbols with text prefix (--pp=prefix) * $onEchoV > def1.gms $if not set pp $set pp set sx / h1,h2 /; $onExternalInput scalar p0 'p0scalar' /1/; set s1 's1set' /%pp%1/; parameter p1 'p1parameter' /%pp%1 1/, p2(*,sx) 'p2parameter' / %pp%1.h1 1 /; table p2; variable x1(*,*) 'x1variable' / %pp%1.%pp%3.L 1 /; equation e1(*) 'e1equation' / %pp%3.L 1 /; $offExternalInput $onExternalOutput scalar op0 'op0scalar' /1/; set os1 'os1set' /1.2/; parameter op1 'op1parameter' /1 1/, op2(*,sx) 'op2parameter' / 1.h1 1 /; table op2; variable ox1(*,*) 'ox1variable' / 1.3.L 1 /; equation oe1(*) 'oe1equation' / 3.L 1 /; $offExternalOutput $offEcho * Create data contract file as well as input and in/out GDX files $call.checkErrorLevel gams def1.gms lo=%gams.lo% IDCGenerateJSON=def1.json IDCGenerateGDXInput=def1orgin.gdx IDCGenerateGDX=def1orginout.gdx * * GAMS Program with external input and output with data via GDX file * Different order of symbols but otherwise identical * $onEchoV > def2.gms set sx / h1,h2 /; $onExternalInput variable x1(*,*) 'x1variable'; equation e1(*) 'e1equation'; scalar p0 'p0scalar'; set s1(*) 's1set'; parameter p%one%(*) 'p1parameter', p2(*,sx) 'p2parameter';table p2; $offExternalInput $onExternalInput $gdxIn %gdxInFile% $load p0 s1 p%one% p2 x1 e1 $offExternalInput $onExternalOutput variable ox1(*,*) 'ox1variable'; equation oe1(*) 'oe1equation'; scalar op0 'op0scalar'; set os1(*,*) 'os1set'; parameter op1(*) 'op1parameter', op2(*,sx) 'op2parameter'; table op2; $offExternalOutput $offEcho $onEchoV > def3.gms set sx / h1,h2 /; $onExternalInput parameter p%one%(*) 'p1parameter', p2(*,sx) 'p2parameter';table p2; variable x1(*,*) 'x1variable'; equation e1(*) 'e1equation'; scalar p0 'p0scalar'; set s1(*) 's1set'; $onEmbeddedCode Python: gams.set('p%one%',[]) gams.set('p2',[]) gams.set('x1',[]) gams.set('e1',[]) gams.set('p0',[15]) $offEmbeddedCode p%one% p2 x1 e1 p0 s1 %system.nullfile% 2>&1 $if not errorlevel 1 $abort json files def1.json def2.json should be different $call gams def2.gms lo=%gams.lo% IDCJSON=def1.json --one=one --gdxInFile=def1orgin.gdx $if not errorlevel 1 $abort data contract should be different * Run def2: Load data implicitely and generate in/out symbols in def2impl.gdx $call.checkErrorLevel gams def2.gms lo=%gams.lo% IDCGDXInput=def1orgin.gdx IDCGenerateGDX=def2impl.gdx --one=1 * Compare the different in/out files, they should be identical $call.checkErrorLevel gdxdiff def2expl.gdx def2impl.gdx > %system.nullfile% * Run def1: Load data implicitely and generate in/out symbols in def1impl.gdx, use different label prefix to have different data statements $call.checkErrorLevel gams def1.gms lo=%gams.lo% IDCGDXInput=def1orgin.gdx IDCGenerateGDX=def1impl.gdx --pp=pp * Compare the different in/out files, they should be identical $call.checkErrorLevel gdxdiff def1impl.gdx def1orginout.gdx > %system.nullfile% $ifThen setEnv GMSPYTHONHOME * Run def3: Create data contract and compare with def1 data contract (IDCJSON) $ call.checkErrorLevel gams def3.gms lo=%gams.lo% IDCGenerateJSON=def3.json IDCJSON=def1.json --one=1 * Compare JSON text files, they shopuld be different due to different symbol order in def1 and def3 $ call diff def1.json def3.json > %system.nullfile% 2>&1 $ if not errorlevel 1 $abort json files def1.json def3.json should be different $ call gams def3.gms lo=%gams.lo% IDCJSON=def1.json --one=one $ if not errorlevel 1 $abort data contract should be different * Run def3: Load data implicitely and generate in/out symbols in def3impl.gdx $ call.checkErrorLevel gams def3.gms lo=%gams.lo% IDCGDXInput=def1orgin.gdx IDCGenerateGDX=def3impl.gdx --one=1 * Compare the different in/out files, they should be identical $ call.checkErrorLevel gdxdiff def2impl.gdx def3impl.gdx > %system.nullfile% $endIf * * Check nesting of $on/offExternalIn/Output * $onEchoV > nl.gms $onExternalOutput $offExternalOutput $onExternalInput scalar a /0/; $offExternalInput $onExternalInput $offExternalInput $onExternalOutput scalar b /1/; $offExternalOutput $onExternalOutput $offExternalOutput $onExternalInput $offExternalInput $offEcho $call.checkErrorLevel gams nl.gms lo=%gams.lo% * * Some runs that should fail * $onEchoV > nl.gms $onExternalInput $onExternalInput scalar a /0/; $offExternalInput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. $onEchoV > nl.gms scalar a /0/; $onExternalInput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. $onEchoV > nl.gms scalar a /0/; $offExternalInput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. $onEchoV > nl.gms $onExternalInput scalar a /0/; $offExternalInput $offExternalInput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. * * Same runs with Output * $onEchoV > nl.gms $onExternalOutput $onExternalOutput scalar a /0/; $offExternalOutput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. $onEchoV > nl.gms scalar a /0/; $onExternalOutput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. $onEchoV > nl.gms scalar a /0/; $offExternalInput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. $onEchoV > nl.gms $onExternalOutput scalar a /0/; $offExternalOutput $offExternalOutput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. * * Overlapping input and output * $onEchoV > nl.gms $onExternalOutput scalar a /0/; $onExternalInput $offExternalOutput scalar a /0/; $offExternalInput $offEcho $call gams nl.gms lo=%gams.lo% $if not errorlevel 1 $abort expect nl.gms to fail. * * Test if $gdxIn, $[bat,lib,sys]include, $call, $hiddenCall are ignored * $call cp -f def1orginout.gdx x.gdx $echo $log hello > x.gms $call cp -f x.gms "%gams.sysdir%x.gms" $call cp -f x.gms "%gams.sysdir%inclib" $onEchoV > ign.gms $onExternalInput $call.checkErrorLevel gams x.gms lo=%gams.lo% $include x.gms $batInclude x.gms $libInclude x.gms $sysInclude x.gms scalar p0; $gdxIn x.gdx $load p0 $ifThen setEnv GMSPYTHONHOME $ onEmbeddedCode Python: %pystatement% $ offEmbeddedCode $endIf $offExternalInput $offEcho $call.checkErrorLevel gams ign.gms --pystatement=pass lo=%gams.lo% $call rm -f x.gms x.gdx "%gams.sysdir%x.gms" "%gams.sysdir%inclib%system.dirsep%x.gms" $call gams ign.gms --pystatement=nonsense lo=%gams.lo% $if not errorlevel 1 $abort 'expect call to fail' $call.checkErrorLevel gams ign.gms --pystatement=nonsense lo=%gams.lo% IDCGDXInput=def1orginout.gdx * * Test that special $load instructions are ignored * $onEchoV > sloadgen.gms set i /1,2/, j /a,b/; parameter p(i,j) / 1.a 1, 2.b 1/; $gdxOut sloadqonly $unLoad p=q $offEcho $call.checkErrorLevel gams sloadgen.gms lo=%gams.lo% gdx=sloadgen $onEchoV > sload.gms $onExternalInput set i,j; parameter p(i,j); $gdxIn sloadqonly $load i %system.nullfile% $onEchoV > sloadec.gms $onExternalInput set i,j; parameter p(i,j); $onEmbeddedCode Python: gams.set('p',[('1','a',1),('2','b',1)]) $offEmbeddedCode i %system.nullfile% $endIf * * Make sure ponly is not enough (does not contain i or j) * $call gams sload.gms lo=%gams.lo% IDCGDXInput=sloadqonly.gdx gdx xxx $if not errorlevel 1 $abort 'expect sload to fail' $call gams sloadec.gms lo=%gams.lo% IDCGDXInput=sloadqonly.gdx gdx xxx $if not errorlevel 1 $abort 'expect sloadec to fail' * * Test that we have an external data statement * $onEchoV > havedata.gms set i /1/; $onExternalInput set i; $offExternalInput $offEcho $call gams havedata.gms lo=%gams.lo% $if not errorlevel 1 $abort 'expect havedata to fail' $onEchoV > havedata.gms $onExternalInput set i; $offExternalInput set i /1/; $offEcho $call gams havedata.gms lo=%gams.lo% $if not errorlevel 1 $abort 'expect havedata to fail' * * Test that we can't overwrite IDCGDXInput while running * $onEchoV > overw.gms set k / k1*k5 /; $gdxOut ext.gdx $unLoad k=i $gdxOut $onExternalInput set i /1/; $offExternalInput display i; $offEcho * Create ext.gdx $call rm -f ext.gdx $call gams overw.gms lo=%gams.lo% IDCGDXInput=ext.gdx $if not errorlevel 1 $abort 'expect havedata to fail' * * Test that explicit and implicit data work (data statement) * $echo set k / k3*k8 /, kk(k) / #k /; > gendata.gms $call.checkErrorLevel gams gendata.gms lo=%gams.lo% gdx=gendata $onEchoV > mr.gms set k / k1*k5 /; $onExternalInput $%onxxx% set k /k3*k%klast%/; $offExternalInput $eval cardk card(k) $ifE %cardk%<>%num% $abort 'wrong cardinality (%cardk%) of k expect %num%' $offEcho $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=8 --klast=8 --onxxx=onmulti $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=6 --klast=8 --onxxx=onmultiR $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=8 --klast=3 --onxxx=onmulti IDCGDXInput=gendata.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=6 --klast=3 --onxxx=onmultiR IDCGDXInput=gendata.gdx * * Test that explicit and implicit data work (GDX $load[M,R]) * $onEchoV > mr.gms set k / k1*k5 /; $onExternalInput set k; $gdxIn gendataX.gdx $%loadxxx% k $offExternalInput $eval cardk card(k) $ifE %cardk%<>%num% $abort 'wrong cardinality (%cardk%) of k expect %num%' $offEcho $call cp -f gendata.gdx gendataX.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=8 --loadxxx=loadM $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=6 --loadxxx=loadR $call rm -f gendataX.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=8 --loadxxx=loadM IDCGDXInput=gendata.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=6 --loadxxx=loadR IDCGDXInput=gendata.gdx * * Test that explicit and implicit data work (GDX $onMulti[R] and $load) * $onEchoV > mr.gms set k / k1*k5 /; $%onxxx% $onExternalInput set k; $gdxIn gendataX.gdx $load k $offExternalInput $eval cardk card(k) $ifE %cardk%<>%num% $abort 'wrong cardinality (%cardk%) of k expect %num%' $offEcho $call cp -f gendata.gdx gendataX.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=8 --onxxx=onmulti $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=6 --onxxx=onmultiR $call rm -f gendataX.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=8 --onxxx=onmulti IDCGDXInput=gendata.gdx $call.checkErrorLevel gams mr.gms lo=%gams.lo% --num=6 --onxxx=onmultiR IDCGDXInput=gendata.gdx * * load obeys filtered/domain checked * $onEchoV > dc.gms set k / k1*k3 /; $onExternalInput set kk(k); $gdxIn gendataX.gdx $%loadxxx% kk $offExternalInput $eval cardkk card(kk) $if not %cardkk%==%num% $abort 'wrong cardinality (%cardkk%) of kk expect %num%' $offEcho $call cp -f gendata.gdx gendataX.gdx $call.checkErrorLevel gams dc.gms lo=%gams.lo% --num=1 --loadxxx=load $call gams dc.gms lo=%gams.lo% --num=UNDF --loadxxx=loaddc $if not errorlevel 1 $abort 'expect dc to fail' $call rm -f gendataX.gdx $call.checkErrorLevel gams dc.gms lo=%gams.lo% --num=1 --loadxxx=load IDCGDXInput=gendata.gdx $call gams dc.gms lo=%gams.lo% --num=UNDF --loadxxx=loaddc IDCGDXInput=gendata.gdx $if not errorlevel 1 $abort 'expect dc to fail' * * Test table layout * $onEchoV > table.gms set k / kHeader1,kHeader2 /; $onExternalInput table t(*,k) kHeader1 r1 1 ; $offExternalInput $offEcho $call.checkErrorLevel gams table.gms lo=%gams.lo% IDCGenerateJSON=table.json $call.checkErrorLevel grep kheader1 table.json > %system.nullfile% * table column index also external input $onEchoV > table.gms $onExternalInput set k / kHeader1,kHeader2 /; table t(*,k) kHeader1 r1 1 ; $offExternalInput $offEcho $call.checkErrorLevel gams table.gms lo=%gams.lo% IDCGenerateJSON=table.json $call grep kheader1 table.json > %system.nullfile% $if not errorlevel 1 $abort 't should not be a table' * table column index single uel $onEchoV > table.gms set k / kHeader1 /; $onExternalInput table t(*,k) kHeader1 r1 1 ; $offExternalInput $offEcho $call.checkErrorLevel gams table.gms lo=%gams.lo% IDCGenerateJSON=table.json $call grep kheader1 table.json > %system.nullfile% $if not errorlevel 1 $abort 't should not be a table' * table column index also implicit external input $onEchoV > table.gms set k; $onExternalInput table t(*,k<) kHeader1 r1 1 ; $offExternalInput $offEcho $call.checkErrorLevel gams table.gms lo=%gams.lo% IDCGenerateJSON=table.json $call grep kheader1 table.json > %system.nullfile% $if not errorlevel 1 $abort 't should not be a table' * table column index is universe $onEchoV > table.gms $onExternalInput table t(*,*) kHeader1 r1 1 ; $offExternalInput $offEcho $call.checkErrorLevel gams table.gms lo=%gams.lo% IDCGenerateJSON=table.json $call grep kheader1 table.json > %system.nullfile% $if not errorlevel 1 $abort 't should not be a table' $label cont * Test --HCUBE stuff $onEchoV > hcube.gms $onExternalInput scalar f / 3 / singleton set ss1 / 'empty' 'nothing' / $offExternalInput $eval f f $if set expectf $if not %f%==%expectf% $abort expect scalar f to be %expectf% $eval.Set SS1TL ss1.tl $eval.Set SS1TE ss1.te $if set expectsstl $if not x%SS1TL%==x%expectsstl% $abort expect singleton set ss1 tl to be %expectsstl% $if set expectsste $if not x%SS1TE%==x%expectsste% $abort expect singleton set ss1 te to be %expectsste% $offEcho $call.checkErrorLevel gamslib -q 1 && gams trnsport lo=0 gdx=hcube.gdx a=c $call.checkErrorLevel gams hcube lo=%gams.lo% --HCUBE_SCALARV_f=15 --expectf=15 --HCUBE_SCALARV_ss1=leer --expectss1=leer --HCUBE_SCALART_ss1=nuescht --expectss1=nuescht idcGDXInput=hcube.gdx $call.checkErrorLevel gams hcube lo=%gams.lo% --expectf=90 --HCUBE_SCALARV_ss1=leer --expectss1=leer --HCUBE_SCALART_ss1=nuescht --expectss1=nuescht idcGDXInput=hcube.gdx * Need to use a pf file because empty string "" is handled funny in Unix shell $onEcho > hcube.pf --HCUBE_SCALARV_ss1="" --expectss1="" --HCUBE_SCALART_ss1="" --expectss1="" idcGDXInput=hcube.gdx $offEcho $call.checkErrorLevel gams hcube lo=%gams.lo% pf=hcube.pf --HCUBE_SCALARV_f=15 --expectf=15 $call.checkErrorLevel gams hcube lo=%gams.lo% pf=hcube.pf --expectf=90 * Check for handling of different domain elements in IDC data and declared data $onEcho > d.gms Scalar x / 6 /; Set j / j3*j5 / jj(j) / j3*j5 /; Set Table jjj(j,j) j5 j5 1 ; $offEcho $onEcho > d2.gms $onExternalInput Set j / j6*j7 / jj(j) / j6,j7 /; Set Table jjj(j,j) j8 j6 1 ; $offExternalInput $offEcho $call.checkErrorLevel gams d lo=%gams.lo% GDX d $call.checkErrorLevel gams d2 lo=%gams.lo% IDCGDXInput d.gdx * Make sure that uels from ignored datastatements get not stored $onEcho > d3.gms $onExternalInput Set j / j5*j10 / jj(j) / j6,j10 /; Set Table jjj(j,j) j5 j10 1 ; $offExternalInput Alias(u,*); abort$(card(u)<>3) 'External data has just 3 uels'; loop(j, if(sameas(j,'j3'), abort$(ord(j)<>1) 'UEL 1 should be j3'); if(sameas(j,'j4'), abort$(ord(j)<>2) 'UEL 2 should be j4'); if(sameas(j,'j5'), abort$(ord(j)<>3) 'UEL 3 should be j5'); ); $offEcho $call.checkErrorLevel gams d3 lo=%gams.lo% IDCGDXInput d.gdx * External input symbols cannot be changed at execution time - by default $onEcho > idcProtect.gms Set i / i1*i6 /; Set j / j1*j3 /; $onExternalInput Scalar x / 5 / y / 8 /; Parameter A(*) / system.empty 0 / b(i,j) / system.empty.system.empty 0 /; Set ij(i,j) / system.empty.system.empty /; $offExternalInput $onMulti x=6; $ifThenE %expectErrors%=1 $if errorfree $abort Assignment to x successful but error expected $clearError $endIf execute_load 'd.gdx', x; $ifThenE %expectErrors%=1 $if errorfree $abort Loading x from GDX successful but error expected $clearError $endIf $ifThen setEnv GMSPYTHONHOME EmbeddedCode Python: gams.set("x", [1]) PauseEmbeddedCode x $ifThenE %expectErrors%=1 $if errorfree $abort Loading x from EC successful but error expected $clearError $endIf EmbeddedCode Python: gams.set("x", [1]) EndEmbeddedCode x $ifThenE %expectErrors%=1 $if errorfree $abort Loading x from EC successful but error expected $clearError $endIf $endIf Option xc; $ifThenE %expectErrors%=1 $if errorfree $abort Permuting into b successful but error expected $clearError $endIf option clear=x; $ifThenE %expectErrors%=1 $if errorfree $abort Clearing x successful but error expected $clearError $endIf option kill=x; $ifThenE %expectErrors%=1 $if errorfree $abort Killing x successful but error expected $clearError $endIf option shuffle = A; $ifThenE %expectErrors%=1 $if errorfree $abort Shuffeling A successful but error expected $clearError $endIf Option ij(i:j); $ifThenE %expectErrors%=1 $if errorfree $abort Index matching for ij successful but error expected $clearError $endIf $offEcho * Run with a=c since we mess a lot with $clearError $call.checkErrorLevel gams idcProtect.gms lo=%gams.lo% a=c --expectErrors=1 * With IDCProtect=0, things should work $call.checkErrorLevel gams idcProtect.gms lo=%gams.lo% IDCProtect=0 --expectErrors=0 * Make sure, that we can continue with an $gdxIn handle in another externalInput block, if no other handle was used in between $onEcho > gdxInCont.gms $onExternalInput $gdxIn d.gdx Set j(*); $load j $offExternalInput $ifThenE %wantError%=1 * New handle outside of external input $gdxIn d.gdx $endIf display j; $onExternalInput Set jj(*); $load jj $offExternalInput $offEcho * This should work without IDCGDXInput $call.checkErrorLevel gams gdxInCont.gms lo=%gams.lo% a=c --wantError=0 * This should work with IDCGDXInput $call.checkErrorLevel gams gdxInCont.gms IDCGDXInput=d.gdx lo=%gams.lo% a=c --wantError=0 * This should fail without IDCGDXInput $call gams gdxInCont.gms lo=%gams.lo% a=c --wantError=1 $if not errorlevel 1 $abort 'Error expected' * This should fail with IDCGDXInput $call gams gdxInCont.gms IDCGDXInput=d.gdx lo=%gams.lo% a=c --wantError=1 $if not errorlevel 1 $abort 'Error expected'