Synchronizing with digestercli and Ant

Synchronizing with digestercli

The Digester software also supports synchronizing from the command-line. You can pass files, directories, or network files to be processed. In theory, all sync operations of the graphical program version can also be done using digestercli, so the software can be called from shell scripts for example.

You can either create your program calls manually or let the Digester GUI version assemble the necessary parameters for you. In the latter case, please configure a sync operation with the sync window as if you would do some synchronizing from there. Once all options are as desired, let the window assemble an appropriate digestercli call by pressing the console symbol at the bottom. This snippet can be easily copied into your own scripts.

All supported options can be passed in any order, but the corresponding values ​​must be in place. All arguments are optional, input errors are detected as far as possible and compensated automatically. Either the program reports incorrect arguments directly or creates appropriate entries in the log file. You can pass files, directories, or URLs in the source and target sets with the -sourceset and -targetset (plus name) arguments.

The command-line version of the Digester software currently provides support for the following arguments and values, which may be printed by calling the application using digestercli -? sync:

 

Usage: digestercli [-options] -sourceset <file|directory|url> ... -targetset <name> <file|directory|url> ...

Supported sync mode options include:

-sync                           synchronize files and directories with message digests
-bsdperdirectory                process BSD-compliant file per directory
-bsdpertarget                   process BSD-compliant file per target
-flipsourcetarget <set>         reverse mode: transfer from target set to source
-gensrctemplate <template>      execute generate template before sync (source set)
-gensrctemplaterefresh          execute template in refresh mode (source set)
-gentemplate <set> <template>   execute generate template before sync (named target set)
-gentemplaterefresh <set>       execute template in refresh mode (named target set)
-gnuperdirectory                process MD5SUM-compliant file per directory
-gnupertarget                   process MD5SUM-compliant file per target
-ignoreregex <value>            regular expression for files to be ignored
-includeregex <value>           regular expression for files to be included
-md2                            process MD2 message digests
-md4                            process MD4 message digests
-md5                            process MD5 message digests
-optdigest                      digests for source and target match
-optfilesize                    source/target file sizes match
-optintermediate                intermediates for source and target match
-optpgpsig                      PGP signatures for source and target match
-opttimestamp                   source/target timestamps match
-optxmlentryfilesize            XML entry file sizes match source and target
-optxmlentrytimestamp           XML entry timestamps match source and target
-overwrite                      overwrite existing files
-pgppertarget                   process PGP signature file per target
-ripemd128                      process RIPEMD-128 hash files
-ripemd160                      process RIPEMD-160 hash files
-ripemd256                      process RIPEMD-256 hash files
-ripemd320                      process RIPEMD-320 hash files
-sha1 | sha                     process SHA-1 hash files
-sha224                         process SHA-224 hash files
-sha256                         process SHA-256 hash files
-sha384                         process SHA-384 hash files
-sha512                         process SHA-512 hash files
-sourcetimestamp                apply source file timestamps to target files
-subdirectories                 include subdirectories
-syncgnubsdtimestamp            trust digests if file timestamp matches (GNU/BSD)
-syncmapping <value>            use sync mapping options as defaults
-syncpgptimestamp               trust signatures if file timestamp matches (PGP)
-syncxmlentryfilesize           trust digests if XML entry file size matches
-syncxmlentrytimestamp          trust digests if XML entry timestamp matches
-syncxmltimestamp               trust digests if file timestamp matches (XML)
-tiger                          process Tiger hash files
-transferall                    transfer all sources without optimization
-verbose                        print additional messages
-whirlpool                      process Whirlpool hash files
-xmlperdirectory                process Digester XML file per directory
-xmlpertarget                   process Digester XML file per target

 

-sync

Activate the operation mode to synchronize files and directories with digests and OpenPGP signatures. The arguments basically are the same as the options shown by the sync window of the graphical version of the Digester software.

If the parameters described below are missing, then the sync window defaults will be used where possible.You can list the currently used arguments and values with the -verbose parameter.

-bsdperdirectory

Check digest files in BSD format for each directory. All digests generated for the original files in a single directory are stored into one single digest file. The algorithms can be specified using the parameter -md5 for MD5 digests etc. The files are named with the extension specified by the -extmd5 parameter etc. and the file name specified with the -summaryfile parameter.

In addition you can let the software check BSD files for each original file (see parameter -bsdpertarget).

-bsdpertarget

Check digest files in BSD format for each original file. For each of the calculated original files one single digest file is being created per algorithm (e.g. -md5 for MD5). The files are named with the extension specified by the -extmd5 parameter etc.

In addition you can let the software check BSD files on a directory basis (see parameter -bsdperdirectory).

-flipsourcetarget <set>

The synchronization normally takes place from source to target, which means that the sources are transferred over to the targets. It may become necessary to transfer files in reverse direction from targets to sources. The target set specified with this parameter ist then used exclusively for transfer. The other targets sets are ignored in reverse mode.

-gensrctemplate <template>

Sets the generate template to be executed for the sources. This template will then be executed right before the sychronization in order to update the digests/signatures for the sources. Optionally the refreshing may be executed (see parameter -gensrctemplaterefresh), if only the new or modified files should be updated.

-gensrctemplaterefresh

Refresh sources instead of calculating them completely in order to update only the new or modified files. This argument is evaluated only if the -gensrctemplate parameter is also specified.

-gentemplate <set> <template>

Sets the generate template (2nd value) to be executed for the named target set (1st value). This template will then be executed right before the sychronization in order to update the digests/signatures for the targets. Optionally the refreshing may be executed (see parameter -gentemplaterefresh ), if only the new or modified files should be updated.

-gentemplaterefresh

Refresh targets instead of calculating them completely in order to update only the new or modified files. This argument is evaluated only if the -gentemplate parameter is also specified.

-gnuperdirectory

Check digest files in GNU format for each directory. All digests generated for the original files in a single directory are stored into one single digest file. The algorithms can be specified using the parameter -md5 for MD5 digests etc. The files are named with the extension specified by the -extmd5 parameter etc. and the file name specified with the -summaryfile parameter.

In addition you can let the software check GNU files for each original file (see parameter -gnupertarget).

-gnupertarget

Check digest files in GNU format for each original file. For each of the calculated original files one single digest file is being created per algorithm (e.g. -md5 for MD5). The files are named with the extension specified by the -extmd5 parameter etc.

In addition you can let the software check GNU files on a directory basis (see parameter -gnuperdirectory).

-ignoreregex <value>

Sets a regular expression to ignore certain files for synchronization (not processed). Alternatively you can include certain files using the parameter -includeregex.

-includeregex <value>

Sets a regular expression to include certain files for synchronization (are being processed). Alternatively you can ignore certain files using the parameter -ignoreregex.

-md2

Activates the recognition of MD2 digests for synchronization.

-md4

Activates the recognition of MD4 digests for synchronization.

-md5

Activates the recognition of MD5 digests for synchronization.

-optdigest

Optimization: do not transfer files if digests match for source and target. Evaluated only if the -transferall argument is not specified.

-optfilesize

Optimization: do not transfer files if the file sizes match for source and target. Evaluated only if the -transferall argument is not specified.

-optintermediate

Optimization: Do not transfer files if the intermediate digests match for source and target (DIgester XML only). Evaluated only if the -transferall argument is not specified.

-optpgpsig

Optimization: do not transfer files if the OpenPGP signatures match for source and target. Evaluated only if the -transferall argument is not specified.

-opttimestamp

Optimization: do not transfer files if the file timestamps match for source and target. Evaluated only if the -transferall argument is not specified.

-optxmlentryfilesize

Optimization: do not transfer files if the XML entry file sizes match for source and target (XML element values). Evaluated only if the -transferall argument is not specified.

-optxmlentrytimestamp

Optimization: do not transfer files if the XML entry timestamps match for source and target (XML element values). Evaluated only if the -transferall argument is not specified.

-overwrite

Existing digest/signature files on the target system are overwritten during synchronization.

You will not be able to restore overwritten digest/signature files with the program, please backup your data first.

-pgppertarget

Check OpenPGP signature files for each original file. For each of the calculated original files one single signature file is being created containing a value for each key. The files are named with the extension specified by the -extpgp parameter.

Creating/checking signature files on a directory basis is not possible for OpenPGP signatures.

-ripemd128

Activates the recognition of RIPEMD-128 digests for synchronization.

-ripemd160

Activates the recognition of RIPEMD-160 digests for synchronization.

-ripemd256

Activates the recognition of RIPEMD-256 digests for synchronization.

-ripemd320

Activates the recognition of RIPEMD-320 digests for synchronization.

-sha1 | sha

Activates the recognition of SHA-1 digests for synchronization.

-sha224

Activates the recognition of SHA-224 digests for synchronization.

-sha256

Activates the recognition of SHA-256 digests for synchronization.

-sha384

Activates the recognition of SHA-384 digests for synchronization.

-sha512

Activates the recognition of SHA-512 digests for synchronization.

-sourcetimestamp

Timestamps of the processed source files are applied to the target files. This is only possible for GNU/BSD/OpenPGP/XML files per target, not for files created on the directory basis (per directory).

-subdirectories

Activates the processing of subdirectories for directory-based synchronization. Otherwise only the specified directories themselves (without subdirectories) are being processed.

-syncgnubsdtimestamp

Trust option: digests is trusted most if GNU/BSD file timestamps match. Those digests/signatures are selected for synchronization of sources and targets which fulfill the maximum number of trust options (is trusted most).

-syncmapping <value>

The values of the given mapping will be used as defaults and overwritten by the specified parameters. By default the options of the current mapping selected in the sync window will be used as defaults.

-syncpgptimestamp

Trust option: signatures is trusted most if OpenPGP file timestamps match. Those digests/signatures are selected for synchronization of sources and targets which fulfill the maximum number of trust options (is trusted most).

-syncxmlentryfilesize

Trust option: digests/signatures is trusted most if the file sizes from within the XML elements match. Those digests/signatures are selected for synchronization of sources and targets which fulfill the maximum number of trust options (is trusted most).

-syncxmlentrytimestamp

Trust option: digests/signatures is trusted most if the file timestamps from within the XML elements match. Those digests/signatures are selected for synchronization of sources and targets which fulfill the maximum number of trust options (is trusted most).

-syncxmltimestamp

Trust option: digests/signatures is trusted most if XML file timestamps match. Those digests/signatures are selected for synchronization of sources and targets which fulfill the maximum number of trust options (is trusted most).

-tiger

Activates the recognition of Tiger digests for synchronization.

-transferall

All sources are transferred without analyzing the targets. If you want to make sure that sources and targets are equal then please choose this option. If not set you can set one or multiple optimization parameters: -optdigest, -optfilesize, -optintermediate, -optpgpsig, -opttimestamp, -optxmlentryfilesize or -optxmlentrytimestamp in order to transfer only a minimum amount of data.

-verbose

Create more detailed messages. Especially all passed arguments and values are presented as a list for control purposes. For many program settings defaults are being used if a parameter is not specified. This functionality is useful if templates are used as default values where the current settings are to be controlled right before the processing starts.

-whirlpool

Activates the recognition of Whirlpool digests for synchronization.

-xmlperdirectory

Check digest/signature files in Digester XML format for each directory. All digests generated for the original files in a single directory are stored into one single digest file. The algorithms can be specified using the parameter -md5 for MD5 digests etc. The files are named with the extension specified by the -extxml parameter and the file name specified with the -summaryfile parameter.

In addition you can let the software check Digester XML files for each original file (see parameter -xmlpertarget).

-xmlpertarget

Check digest/signature files in Digester XML format for each original file. For each of the calculated original files one single XML file is being created containing the values for the chosen algorithms (e.g. -md5 for MD5) as well as the OpenPGP signatures. The files are named with the extension specified by the -extxml parameter.

In addition you can let the software check XML files on a directory basis (see parameter -xmlperdirectory).

 

Synchronizing with Ant

The Digester software also supports synchronizing from Ant build scripts. You can pass files, directories, or network files to be processed. In theory, all sync operations of the graphical program version can also be done using Apache Ant, so the software can be called from own builds scripts for example.

You can either create your XML tags manually or let the Digester GUI version assemble the necessary elements and attributes for you. In the latter case, please configure a sync operation with the sync window as if you would do some synchronizing from there. Once all options are as desired, let the window assemble an appropriate Ant build script code by pressing the ant symbol at the bottom. This snippet can be easily copied into your own build scripts.

All supported options can be passed in any order, but the corresponding values ​​must be in place. All arguments are optional, input errors are detected as far as possible and compensated automatically. The program reports incorrect arguments directly and terminates if you have specified the failonerror tag.

For the synchronizing with Ant all arguments of the command-line version of the Digester software digestercli can be used which are represented by separate XML elements each. In the header of your build script the task digestercli must be declared. Please adjust the classpath attribute by the value appropriate for your installation. The Web Start version cannot be used for checking from Ant build scripts.

<taskdef name="digestercli"
   classname="com.lf.digester.ant.DigesterCLITask"
   classpath="C:\Programme\digester-1.6.3\ant\digester-ant.jar"/>

An example XML code for Ant build scripts looks like the following where a server directory is being synchronized with a local directory:

<digestercli>
   <sync />
   <optfilesize />
   <optxmlentryfilesize />
   <optxmlentrytimestamp />
   <optdigest />
   <optintermediate />
   <sourcetimestamp />
   <overwrite />
   <subdirectories />
   <syncgnubsdtimestamp />
   <syncpgptimestamp />
   <syncxmltimestamp />
   <syncxmlentryfilesize />
   <syncxmlentrytimestamp />
   <xmlperdirectory />
   <md5 />
   <sha1 />
   <sourceset>
       <gentemplate>My Sync4</gentemplate>
       <gentemplaterefresh />
       <dirset dir="C:\Temp\digester">
           <include name="cms" />
       </dirset>
   </sourceset>
   <targetset>
       <name>Targets</name>
       <url>sftp://mydomain.com:22/home/user1/public_html/.cms</url>
   </targetset>
</digestercli>

 

Additional Links