#!perl

# Note: This script is a CLI for Riap function /App/UniqFiles/uniq_files
# and generated automatically using Perinci::CmdLine::Gen version 0.496

our $AUTHORITY = 'cpan:PERLANCAR'; # AUTHORITY
our $DATE = '2020-05-28'; # DATE
our $DIST = 'App-UniqFiles'; # DIST
our $VERSION = '0.131'; # VERSION

use 5.010001;
use strict;
use warnings;
use Log::ger;

use Perinci::CmdLine::Any;

my $cmdline = Perinci::CmdLine::Any->new(
    url => "/App/UniqFiles/uniq_files",
    program_name => "uniq-files",
    log => 1,
    read_config => 0,
    read_env => 0,
);

$cmdline->run;

# ABSTRACT: Report or omit duplicate file contents
# PODNAME: uniq-files

__END__

=pod

=encoding UTF-8

=head1 NAME

uniq-files - Report or omit duplicate file contents

=head1 VERSION

This document describes version 0.131 of uniq-files (from Perl distribution App-UniqFiles), released on 2020-05-28.

=head1 SYNOPSIS

Usage:

 % uniq-files [-A=s] [-D] [-R] [-a] [--algorithm=s] [-c] [--count] [-d] [--debug] [--digest-args-json=s] [--digest-args=s] [--format=name] [--group-by-digest] [--json] [--log-level=level] [--(no)naked-res] [--no-count] [--no-group-by-digest] [--no-recurse] [--no-report-unique] [--no-show-digest] [--no-show-size] [--nocount] [--nogroup-by-digest] [--norecurse] [--noreport-unique] [--noshow-digest] [--noshow-size] [--page-result[=program]] [--quiet] [--recurse] [--report-duplicate=s] [--report-unique] [--show-digest] [--show-size] [--trace] [-u] [--verbose] [--view-result[=program]] <files> ...

Examples:

List all files which do no have duplicate contents:

 % uniq-files *

List all files which have duplicate contents:

 % uniq-files -d *

Move all duplicate files (except one copy) in this directory (and subdirectories) to .dupes/:

 % uniq-files -D -R * | while read f; do mv "$f" .dupes/; done

List number of occurences of contents for duplicate files:

 % uniq-files -c *

List number of occurences of contents for all files:

 % uniq-files -a -c *

List all files, along with their number of content occurrences and content digest. Use the BLAKE2b digest algorithm. And group the files according to their digest.:

 % uniq-files -a -c --show-digest -A BLAKE2,blake2b *

=head1 DESCRIPTION

Given a list of filenames, will check each file size and content for duplicate
content. Interface is a bit like the C<uniq> Unix command-line program.

=head1 OPTIONS

C<*> marks required options.

=head2 Main options

=over

=item B<--algorithm>=I<s>

What algorithm is used to compute the digest of the content.

The default is to use `md5`. Some algorithms supported include `crc32`, `sha1`,
`sha256`, as well as `Digest` to use Perl <pm:Digest> which supports a lot of
other algorithms, e.g. `SHA-1`, `BLAKE2b`.

If set to '', 'none', or 'size', then digest will be set to file size. This
means uniqueness will be determined solely from file size. This can be quicker
but will generate a false positive when two files of the same size are deemed as
duplicate even though their content may be different.


=item B<--count>, B<-c>

Whether to return each file content's number of occurence.

1 means the file content is only encountered once (unique), 2 means there is one
duplicate, and so on.


=item B<--digest-args-json>=I<s>, B<-A>

See C<--digest-args>.

=item B<--digest-args>=I<s>

Some Digest algorithms require arguments, you can pass them here.


=item B<--files-json>=I<s>

See C<--files>.

=item B<--files>=I<s@>*

Can be specified multiple times.

=item B<--group-by-digest>

Sort files by its digest (or size, if not computing digest), separate each different digest.

=item B<--no-report-unique>

=item B<--recurse>, B<-R>

If set to true, will recurse into subdirectories.


=item B<--report-duplicate>=I<s>

Whether to return duplicate items.

Default value:

 2

Valid values:

 [0,1,2,3]

Can be set to either 0, 1, 2.

If set to 2 (the default), will only return the first of duplicate items. For
example: `file1` contains text 'a', `file2` 'b', `file3` 'a'. Only `file1` will
be returned because `file2` is unique and `file3` contains 'a' (already
represented by `file1`).

If set to 1, will return all the the duplicate files. From the above example:
`file1` and `file3` will be returned.

If set to 3, will return all but the first of duplicate items. From the above
example: `file3` will be returned. This is useful if you want to keep only one
copy of the duplicate content. You can use the output of this routine to `mv` or
`rm`.

If set to 0, duplicate items will not be returned.


=item B<--show-digest>

Show the digest value (or the size, if not computing digest) for each file.

Note that this routine does not compute digest for files which have unique
sizes, so they will show up as empty.


=item B<--show-size>

Show the size for each file.

=item B<-a>

Alias for --report-unique --report-duplicate=1 (report all files).

See C<--no-report-unique>.

=item B<-d>

Alias for --noreport-unique --report-duplicate=1.

See C<--no-report-unique>.

=item B<-D>

Alias for --noreport-unique --report-duplicate=3.

See C<--no-report-unique>.

=item B<-u>

Alias for --report-unique --report-duplicate=0.

See C<--no-report-unique>.

=back

=head2 Logging options

=over

=item B<--debug>

Shortcut for --log-level=debug.

=item B<--log-level>=I<s>

Set log level.

=item B<--quiet>

Shortcut for --log-level=error.

=item B<--trace>

Shortcut for --log-level=trace.

=item B<--verbose>

Shortcut for --log-level=info.

=back

=head2 Output options

=over

=item B<--format>=I<s>

Choose output format, e.g. json, text.

Default value:

 undef

=item B<--json>

Set output format to json.

=item B<--naked-res>

When outputing as JSON, strip result envelope.

Default value:

 0

By default, when outputing as JSON, the full enveloped result is returned, e.g.:

    [200,"OK",[1,2,3],{"func.extra"=>4}]

The reason is so you can get the status (1st element), status message (2nd
element) as well as result metadata/extra result (4th element) instead of just
the result (3rd element). However, sometimes you want just the result, e.g. when
you want to pipe the result for more post-processing. In this case you can use
`--naked-res` so you just get:

    [1,2,3]


=item B<--page-result>

Filter output through a pager.

=item B<--view-result>

View output using a viewer.

=back

=head2 Other options

=over

=item B<--help>, B<-h>, B<-?>

Display help message and exit.

=item B<--version>, B<-v>

Display program's version and exit.

=back

=head1 COMPLETION

This script has shell tab completion capability with support for several
shells.

=head2 bash

To activate bash completion for this script, put:

 complete -C uniq-files uniq-files

in your bash startup (e.g. F<~/.bashrc>). Your next shell session will then
recognize tab completion for the command. Or, you can also directly execute the
line above in your shell to activate immediately.

It is recommended, however, that you install modules using L<cpanm-shcompgen>
which can activate shell completion for scripts immediately.

=head2 tcsh

To activate tcsh completion for this script, put:

 complete uniq-files 'p/*/`uniq-files`/'

in your tcsh startup (e.g. F<~/.tcshrc>). Your next shell session will then
recognize tab completion for the command. Or, you can also directly execute the
line above in your shell to activate immediately.

It is also recommended to install L<shcompgen> (see above).

=head2 other shells

For fish and zsh, install L<shcompgen> as described above.

=head1 HOMEPAGE

Please visit the project's homepage at L<https://metacpan.org/release/App-UniqFiles>.

=head1 SOURCE

Source repository is at L<https://github.com/perlancar/perl-App-UniqFiles>.

=head1 BUGS

Please report any bugs or feature requests on the bugtracker website L<https://rt.cpan.org/Public/Dist/Display.html?Name=App-UniqFiles>

When submitting a bug or request, please include a test-file or a
patch to an existing test-file that illustrates the bug or desired
feature.

=head1 AUTHOR

perlancar <perlancar@cpan.org>

=head1 COPYRIGHT AND LICENSE

This software is copyright (c) 2020, 2019, 2017, 2015, 2014, 2012, 2011 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.

=cut
