Refreshing with digestercli and Ant

Refreshing with digestercli

The Digester software also supports refreshing from the command-line. You can pass files, directories, or network files to be processed. In theory, all refresh operations of the graphical program version can also be done using digestercli, so the software can be called from shell scripts for example.

You can either create your program calls manually or let the Digester GUI version assemble the necessary parameters for you. In the latter case, please configure a refresh operation with the generate window as if you would do some refreshing from there. Once all options are as desired, let the window assemble an appropriate digestercli call by pressing the console symbol at the bottom. This snippet can be easily copied into your own scripts.

All supported options can be passed in any order, but the corresponding values ​​must be in place. All arguments are optional, input errors are detected as far as possible and compensated automatically. Either the program reports incorrect arguments directly or creates appropriate entries in the log file. You can pass files, directories, or URLs to be processed as a space-separated list using the -targets argument.

The command-line version of the Digester software currently provides support for the following arguments and values, which may be printed by calling the application using digestercli -? refresh:

 

Usage: digestercli [-options] -targets <file|directory|url> ...

Supported refresh mode options include:

-refresh                        refresh message digest files
-bsdperdirectory                process BSD-compliant file per directory
-bsdpertarget                   process BSD-compliant file per target
-exponentialstart <value>       start creating XML intermediates at byte index
-extmd2 <value>                 file extension for MD2
-extmd4 <value>                 file extension for MD4
-extmd5 <value>                 file extension for MD5
-extpgp <value>                 file extension for PGP signatures
-extripemd128 <value>           file extension for RIPEMD-128
-extripemd160 <value>           file extension for RIPEMD-160
-extripemd256 <value>           file extension for RIPEMD-256
-extripemd320 <value>           file extension for RIPEMD-320
-extsha1 <value>                file extension for SHA-1
-extsha224 <value>              file extension for SHA-224
-extsha256 <value>              file extension for SHA-256
-extsha384 <value>              file extension for SHA-384
-extsha512 <value>              file extension for SHA-512
-exttiger <value>               file extension for Tiger
-extwhirlpool <value>           file extension for Whirlpool
-extxml <value>                 file extension for Digester XML
-gnuperdirectory                process MD5SUM-compliant file per directory
-gnupertarget                   process MD5SUM-compliant file per target
-hexcase <value>                format of hexadecimal digests (lower|upper)
-ignoredigests                  do not generate digests for digest files
-ignoreregex <value>            regular expression for files to be ignored
-includeregex <value>           regular expression for files to be included
-maxintermediates <value>       limit the number of XML intermediates
-md2                            process MD2 message digests
-md4                            process MD4 message digests
-md5                            process MD5 message digests
-overwrite                      overwrite existing files
-pgpformat <value>              write binary or ASCII PGP signature files (ascii|binary)
-pgpkey <id> <url>              use PGP key with ID (required) and URL (optional)
-pgppertarget                   process PGP signature file per target
-pgpversion <value>             version header for PGP signatures
-refreshallformats              search files in all formats for existing digests
-refreshremoveother             delete message digest files in other formats
-refreshremoveartifacts         delete message digest files without targets
-refreshignoremissing           recycle digest although intermediates are missing
-refreshgnubsdtimestamp         recycle digest if GNU/BSD file timestamp matches
-refreshpgptimestamp            recycle digest if PGP signature file timestamp matches
-refreshxmltimestamp            recycle digest if XML file timestamp matches
-refreshxmlentryfilesize        recycle digest if target XML entry file size matches
-refreshxmlentrytimestamp       recycle digest if target XML entry timestamp matches
-ripemd128                      process RIPEMD-128 hash files
-ripemd160                      process RIPEMD-160 hash files
-ripemd256                      process RIPEMD-256 hash files
-ripemd320                      process RIPEMD-320 hash files
-sha1 | sha                     process SHA-1 hash files
-sha224                         process SHA-224 hash files
-sha256                         process SHA-256 hash files
-sha384                         process SHA-384 hash files
-sha512                         process SHA-512 hash files
-subdirectories                 include subdirectories
-summaryfile <value>            name of the directory files without extension
-template <value>               use template options as defaults
-tiger                          process Tiger hash files
-timestamp                      apply target file timestamps for generated files
-verbose                        print additional messages
-whirlpool                      process Whirlpool hash files
-xmlabsolutepaths               add absolute paths for Digester XML files
-xmladdurls                     add URL elements for Digester XML files
-xmlcomment <value>             comment for Digester XML files
-xmldigestformat <value>        format of XML file digests (hex|base64)
-xmlintermediates <value>       create XML intermediates (linear|exponential)
-xmllinearevery <value>         create XML intermediates every n bytes
-xmlperdirectory                process Digester XML file per directory
-xmlpertarget                   process Digester XML file per target
-xmlpgp                         add PGP signatures for Digester XML files
-xmlpgpkeyid                    add PGP key IDs for Digester XML files
-xmlpgpkeyname                  add PGP key names for Digester XML files
-xmlpgpkeyemail                 add PGP key eMail addresses for Digester XML files
-xmlpgpkeyurl                   add PGP key network addresses for Digester XML files
-xmlurl <value>                 URL address to be added, see xmladdurls

 

-refresh

Activate the operation mode to refresh digests and OpenPGP signatures. The arguments basically are the same as the options shown by the generate window of the graphical version of the Digester software.

If the parameters described below are missing, then the generate window defaults will be used where possible.You can list the currently used arguments and values with the -verbose parameter.

-bsdperdirectory

Create digest files in BSD format for each directory. All digests generated for the original files in a single directory are stored into one single digest file. The algorithms can be specified using the parameter -md5 for MD5 digests etc. The files are named with the extension specified by the -extmd5 parameter etc. and the file name specified with the -summaryfile parameter. Alternatively you can use the -gnuperdirectory argument but not both since the same files are being created.

In addition you can let the software create BSD files for each original file (see parameter -bsdpertarget).

-bsdpertarget

Create digest files in BSD format for each original file. For each of the calculated original files one single digest file is being created per algorithm (e.g. -md5 for MD5). The files are named with the extension specified by the -extmd5 parameter etc. Alternatively you can use the -gnupertarget argument but not both since the same files are being created.

In addition you can let the software create BSD files on a directory basis (see parameter -bsdperdirectory).

-exponentialstart <value>

This option allows intermediate digests to be created starting at a certain position (e.g. 8K = 8 kilobytes). The next intermediates will then be extracted in steps after the next 16 kilobytes, 32 kilobytes etc. You can use numeric values or abbreviations like (K)ilobytes, (M)egabytes, (G)igabytes and (T)erabytes.

This argument is evaluated only if intermediate digests for Digester XML are being calculated (see parameter -xmlintermediates) and if the exponential mode has been specified.

-extmd2 <value>

Sets the file extension for GNU/BSD files containing MD2 message digests (default: .md2). Please do not use the following characters: \ / : * ? " < > |

-extmd4 <value>

Sets the file extension for GNU/BSD files containing MD4 message digests (default: .md4). Please do not use the following characters: \ / : * ? " < > |

-extmd5 <value>

Sets the file extension for GNU/BSD files containing MD5 message digests (default: .md5). Please do not use the following characters: \ / : * ? " < > |

-extpgp <value>

Sets the file extension for signature files containing OpenPGP signatures (default: .asc). Please do not use the following characters: \ / : * ? " < > |

-extripemd128 <value>

Sets the file extension for GNU/BSD files containing RIPEMD-128 message digests (default: .rip128). Please do not use the following characters: \ / : * ? " < > |

-extripemd160 <value>

Sets the file extension for GNU/BSD files containing RIPEMD-160 message digests (default: .rip160). Please do not use the following characters: \ / : * ? " < > |

-extripemd256 <value>

Sets the file extension for GNU/BSD files containing RIPEMD-256 message digests (default: .rip256). Please do not use the following characters: \ / : * ? " < > |

-extripemd320 <value>

Sets the file extension for GNU/BSD files containing RIPEMD-320 message digests (default: .rip320). Please do not use the following characters: \ / : * ? " < > |

-extsha1 <value>

Sets the file extension for GNU/BSD files containing SHA-1 message digests (default: .sha). Please do not use the following characters: \ / : * ? " < > |

-extsha224 <value>

Sets the file extension for GNU/BSD files containing SHA-224 message digests (default: .sha224). Please do not use the following characters: \ / : * ? " < > |

-extsha256 <value>

Sets the file extension for GNU/BSD files containing SHA-256 message digests (default: .sha256). Please do not use the following characters: \ / : * ? " < > |

-extsha384 <value>

Sets the file extension for GNU/BSD files containing SHA-384 message digests (default: .sha384). Please do not use the following characters: \ / : * ? " < > |

-extsha512 <value>

Sets the file extension for GNU/BSD files containing SHA-512 message digests (default: .sha512). Please do not use the following characters: \ / : * ? " < > |

-exttiger <value>

Sets the file extension for GNU/BSD files containing Tiger message digests (default: .tiger). Please do not use the following characters: \ / : * ? " < > |

-extwhirlpool <value>

Sets the file extension for GNU/BSD files containing Whirlpool message digests (default: .wpool). Please do not use the following characters: \ / : * ? " < > |

-extxml <value>

Sets the file extension for Digester XML files containing digests/signatures (default: .digest). Please do not use the following characters: \ / : * ? " < > |

-gnuperdirectory

Create digest files in GNU format for each directory. All digests generated for the original files in a single directory are stored into one single digest file. The algorithms can be specified using the parameter -md5 for MD5 digests etc. The files are named with the extension specified by the -extmd5 parameter etc. and the file name specified with the -summaryfile parameter. Alternatively you can use the -bsdperdirectory argument but not both since the same files are being created.

In addition you can let the software create GNU files for each original file (see parameter -gnupertarget).

-gnupertarget

Create digest files in GNU format for each original file. For each of the calculated original files one single digest file is being created per algorithm (e.g. -md5 for MD5). The files are named with the extension specified by the -extmd5 parameter etc. Alternatively you can use the -bsdpertarget argument but not both since the same files are being created.

In addition you can let the software create GNU files on a directory basis (see parameter -gnuperdirectory).

-hexcase <value>

Sets the case of writing of hexadecimal digest values (lower = lower-case, upper = upper-case).

-ignoredigests

Files containing digests/signatures are not used to calculate additional digests/signatures. This means that no MD5 files are being created for existing MD5 files etc. This argument should be used by default.

-ignoreregex <value>

Sets a regular expression to ignore certain files when searching for digests/signatures (not processed). Alternatively you can include certain files using the parameter -includeregex.

-includeregex <value>

Sets a regular expression to include certain files when searching for digests/signatures (are being processed). Alternatively you can ignore certain files using the parameter -ignoreregex.

-maxintermediates <value>

Use this option to limit the number of calculated intermediate digests.

This argument is evaluated only if intermediate digests for Digester XML are being calculated (see parameter -xmlintermediates) and for both the exponential and linear mode.

-md2

Activates the generation of MD2 message digests. The files are created based on the configured file extensions (see parameter -extmd2).

-md4

Activates the generation of MD4 message digests. The files are created based on the configured file extensions (see parameter -extmd4).

-md5

Activates the generation of MD5 message digests. The files are created based on the configured file extensions (see parameter -extmd5).

-overwrite

Existing digest/signature files are overwritten when the calculation has been finished.

You will not be able to restore overwritten digest/signature files with the program, please backup your data first.

-pgpformat <value>

Specify ascii here to create OpenPGP files in ASCII format (text) or binary for the binary format.

This argument is evaluated only if OpenPGP signatures are being calculated.

-pgpkey <id> <url>

Use the OpenPGP key with the given ID (required) and the key URL (optional) to calculate signatures. The ID must be a full hexadecimal identifier like 0x2452F30BFCE8C30F in order to identify the key. The URL value is optional and used if the additional -xmlpgpkeyurl parameter has been specified (only for Digester XML).

This argument is evaluated only if OpenPGP signatures are being calculated.

-pgppertarget

Create OpenPGP signature files for each original file. For each of the calculated original files one single signature file is being created containing a value for each key. The files are named with the extension specified by the -extpgp parameter.

Creating signature files on a directory basis is not possible for OpenPGP signatures.

-pgpversion <value >

In order to label OpenPGP signatures files one can specify an arbitrary single-line text to be stored in the signature file headers. Here, you can set your company's or organization's name for example.

-refreshallformats

For the refreshing, all detected digest/signature files are being used. If this parameter is omitted then only those formats are being analyzed which are configured to be written and deemed relevant for the synchronization.

This option can be extended with the -refreshremoveother und -refreshremoveartifacts parameters.

-refreshremoveother

If all formats are being analyzed (see parameter -refreshallformats) then it is possible to switch from GNU files over to XML files for example and let the software delete the previous GNU files programmatically. This way only those files are kept which are deemed relevant for the synchronization (write formats).

You will not be able to restore overwritten digest/signature files with the program, please backup your data first.

-refreshremoveartifacts

If all formats are being analyzed (see parameter -refreshallformats) then it is also possible to delete artifact files automatically. Artifacts are files with digests/signatures where no original files exist at all. This way useless files can be cleared automatically.

You will not be able to restore overwritten digest/signature files with the program, please backup your data first.

-refreshignoremissing

Skip file and do not re-calculate if digests match but intermediate digests are missing. Maybe intermediates have not been generated the last time so this can be compensated here as long as the overall digests match for the files.

-refreshgnubsdtimestamp

Skip file and do not re-calculate if digests match and if the file timestamps match.

-refreshpgptimestamp

Skip file and do not re-calculate if signatures match and if the file timestamps match.

-refreshxmltimestamp

Skip file and do not re-calculate if digests/signatures match and if the XML file timestamps match. The entries from within the XML files may be controlled separately with the -refreshxmlentryfilesize and -refreshxmlentrytimestamp arguments.

-refreshxmlentryfilesize

Skip file and do not re-calculate if digests/signatures match and if the file sizes within the XML entries match. The timestamp of the XML file itself may be controlled separately with the -refreshxmltimestamp argument.

-refreshxmlentrytimestamp

Skip file and do not re-calculate if digests/signatures match and if the timestamps within the XML entries match. The timestamp of the XML file itself may be controlled separately with the -refreshxmltimestamp argument.

-ripemd128

Activates the generation of RIPEMD-128 message digests. The files are created based on the configured file extensions (see parameter -extripemd128).

-ripemd160

Activates the generation of RIPEMD-160 message digests. The files are created based on the configured file extensions (see parameter -extripemd160).

-ripemd256

Activates the generation of RIPEMD-256 message digests. The files are created based on the configured file extensions (see parameter -extripemd256).

-ripemd320

Activates the generation of RIPEMD-320 message digests. The files are created based on the configured file extensions (see parameter -extripemd320).

-sha1 | sha

Activates the generation of SHA-1 message digests. The files are created based on the configured file extensions (see parameter -extsha1).

-sha224

Activates the generation of SHA-224 message digests. The files are created based on the configured file extensions (see parameter -extsha224).

-sha256

Activates the generation of SHA-256 message digests. The files are created based on the configured file extensions (see parameter -extsha256).

-sha384

Activates the generation of SHA-384 message digests. The files are created based on the configured file extensions (see parameter -extsha384).

-sha512

Activates the generation of SHA-512 message digests. The files are created based on the configured file extensions (see parameter -extsha512).

-subdirectories

Activates the processing of subdirectories for directory-based generating. Otherwise only the specified directories themselves (without subdirectories) are being processed.

-summaryfile <value>

File names for digest files on the directory basis (default: summary). The file extensions are then chosen according to the format of the digest files (for example see parameter -extxml).

-template <value>

The values of the given template will be used as defaults and overwritten by the specified parameters. By default the options of the current template selected in the generate window will be used as defaults.

-tiger

Activates the generation of Tiger message digests. The files are created based on the configured file extensions (see parameter -exttiger).

-timestamp

Timestamps of the processed original files are applied to the created digest/signature files. This is only possible for GNU/BSD/OpenPGP/XML files per target, not for files created on the directory basis (per directory).

-verbose

Create more detailed messages. Especially all passed arguments and values are presented as a list for control purposes. For many program settings defaults are being used if a parameter is not specified. This functionality is useful if templates are used as default values where the current settings are to be controlled right before the processing starts.

-whirlpool

Activates the generation of Whirlpool message digests. The files are created based on the configured file extensions (see parameter -extwhirlpool).

-xmlabsolutepaths

Special option for Digester XML files: add absolute paths of the processed original files. With this option all digests hold the absolute reference on the computer, e.g. C:\Temp\franz.txt under Window. Even after copying such XML files to another drive of the same computer these paths may be resolved properly allowing to find the original files for the digests. By using this option the Digester XML files become slightly bigger.

-xmladdurls

Spezielle Option für Digester XML-Dateien: Feste Netzwerk-Adresse oder SSI-Ausdruck pro Originaldatei speichern. Der Wert wird mit dem Parameter -xmlurl übergeben.

-xmlcomment <value>

Optional text for the XML file header. This argument is evaluated only if Digester XML files are being generated. Use the comment to deposit your own name or your company's name in order to label the XML files.

-xmldigestformat <value>

Set the encoding format of hexadecimal digests in Digester XML files (hex = hexadecimal, base64 = Base64 encoding). See the -hexcase parameter to control the case of writing (lower/upper) of hexadecimal values.

-xmlintermediates <value>

Create intermediate message digests, deactivated otherwise. Use the linear value to extract intermediates in fixed intervals or exponential to extract intermediates dynamically after blocks like 8 kilobytes, then 16 kilobytes etc.

Additional parameters to control the intermediate digest generation are -xmllinearevery, -maxintermediates and -exponentialstart.

-xmllinearevery <value>

This option allows intermediate digests to be extracted after fixed intervals. The interval size can be specified with values like 10M where intermediates are extracted every 10 megabytes. You can use numeric values or abbreviations like (K)ilobytes, (M)egabytes, (G)igabytes and (T)erabytes.

This argument is evaluated only if intermediate digests for Digester XML are being calculated (see parameter -xmlintermediates) and if the linear mode has been specified.

-xmlperdirectory

Create digest/signature files in Digester XML format for each directory. All digests generated for the original files in a single directory are stored into one single digest file. The algorithms can be specified using the parameter -md5 for MD5 digests etc. The files are named with the extension specified by the -extxml parameter and the file name specified with the -summaryfile parameter.

In addition you can let the software create Digester XML files for each original file (see parameter -xmlpertarget).

-xmlpertarget

Create digest/signature files in Digester XML format for each original file. For each of the calculated original files one single XML file is being created containing the values for the chosen algorithms (e.g. -md5 for MD5) as well as the OpenPGP signatures. The files are named with the extension specified by the -extxml parameter.

In addition you can let the software create XML files on a directory basis (see parameter -xmlperdirectory).

-xmlpgp

If Digester XML files are being created and OpenPGP keys have been specified to calcuate signatures then the signature values may be added to the Digester XML files. Signatures are represented by separate XML elements which are stored on the level together with digests (siehe DTD Digester XML).

-xmlpgpkeyid

In addition to the OpenPGP signatures themselves Digester XML files can contain the IDs of the OpenPGP keys used for signature generation. The IDs can help importing software to identify or download appropriate public keys to check the signatures (siehe Parameter -xmlpgp).

-xmlpgpkeyname

In addition to the OpenPGP signatures themselves Digester XML files can contain the names of the OpenPGP keys used for signature generation (siehe Parameter -xmlpgp).

-xmlpgpkeyemail

In addition to the OpenPGP signatures themselves Digester XML files can contain the eMail addresses of the OpenPGP keys used for signature generation (siehe Parameter -xmlpgp).

-xmlpgpkeyurl

In addition to the OpenPGP signatures themselves Digester XML files can contain the network addresses of the OpenPGP keys used for signature generation. The URLs can help importing software to identify or download appropriate public keys to check the signatures (siehe Parameter -xmlpgp).

-xmlurl <value>

Special option for Digester XML files: add fixed network address or SSI pattern. Here you can specify a static URL pattern e.g. for SSI (Server Side Includes) which can be processed by web servers like the Apache HTTP server. Such URLs can be translated dynamically into the appropriate URLs for the original files on the server. If Digester XML files are being downloaded with the software, then the referenced files can be identified using the URL entries to be downloaded, checked, and processed in other ways.

This argument is evaluated only if also -xmladdurls is passed.

 

Refreshing with Ant

The Digester software also supports refreshing from Ant build scripts. You can pass files, directories, or network files to be processed. In theory, all refresh operations of the graphical program version can also be done using Apache Ant, so the software can be called from own builds scripts for example.

You can either create your XML tags manually or let the Digester GUI version assemble the necessary elements and attributes for you. In the latter case, please configure a refresh operation with the generate window as if you would do some refreshing from there. Once all options are as desired, let the window assemble an appropriate Ant build script code by pressing the ant symbol at the bottom. This snippet can be easily copied into your own build scripts.

All supported options can be passed in any order, but the corresponding values ​​must be in place. All arguments are optional, input errors are detected as far as possible and compensated automatically. The program reports incorrect arguments directly and terminates if you have specified the failonerror tag.

For the refreshing with Ant all arguments of the command-line version of the Digester software digestercli can be used which are represented by separate XML elements each. In the header of your build script the task digestercli must be declared. Please adjust the classpath attribute by the value appropriate for your installation. The Web Start version cannot be used for checking from Ant build scripts.

<taskdef name="digestercli"
   classname="com.lf.digester.ant.DigesterCLITask"
   classpath="C:\Programme\digester-1.6.3\ant\digester-ant.jar"/>

An example XML code for Ant build scripts looks like the following where a directory including subdirectories is being refreshed:

<digestercli>
   <refresh />
   <md5 />
   <extmd5>.md5</extmd5>
   <sha1 />
   <extsha1>.sha</extsha1>
   <xmlperdirectory />
   <overwrite />
   <timestamp />
   <summaryfile>summary</summaryfile>
   <ignoredigests />
   <subdirectories />
   <pgpversion>Digester 1.6.3</pgpversion>
   <extxml>.digest</extxml>
   <xmldigestformat>hex</xmldigestformat>
   <xmlabsolutepaths />
   <xmlpgp />
   <xmlpgpkeyid />
   <xmlpgpkeyname />
   <xmlpgpkeyemail />
   <xmlintermediates>exponential</xmlintermediates>
   <exponentialstart>8K</exponentialstart>
   <maxintermediates>5</maxintermediates>
   <refreshignoremissing />
   <refreshxmlentryfilesize />
   <refreshxmlentrytimestamp />
   <pgpkey>
       <id>0x7EB2B10924E58CDB</id>
   </pgpkey>
   <dirset dir="D:\">
       <include name="checksum_test" />
   </dirset>
</digestercli>
 

 

Additional Links