Difference between revisions of "Xz"

From wikieduonline
Jump to navigation Jump to search
 
(38 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
<code>[[wikipedia:XZ_Utils|xz]]</code><ref>https://man.cx/xz</ref> [[compress]] or decompress (<code>unxz</code>) .xz and .lzma files but xz compresses by default to xz format.
+
{{lc}}
 +
<code>[[wikipedia:XZ_Utils|xz utils]]</code><ref>https://man.cx/xz</ref> (previously [[LZMA]] Utils) [[compress]] or decompress (<code>[[unxz]]</code>) <code>.xz</code> and <code>.lzma</code> files but xz compresses by default to xz format.
 +
 
 +
* <code>[[brew install xz]]</code>
  
 
xz support multi-threaded compression (<code>-T</code> flag)<ref>https://man.cx/xz</ref> since 2014, version 5.2.0. <ref>https://git.tukaani.org/?p=xz.git;a=blob;f=NEWS;hb=HEAD</ref>, as of 2019 threaded decompression hasn’t been implemented yet. <ref>https://man.cx/xz</ref>. Number of threads can be less that defined is file is not big enough for threading with the given settings or if using more threads would exceed the memory usage limit. <ref>https://man.cx/xz</ref>
 
xz support multi-threaded compression (<code>-T</code> flag)<ref>https://man.cx/xz</ref> since 2014, version 5.2.0. <ref>https://git.tukaani.org/?p=xz.git;a=blob;f=NEWS;hb=HEAD</ref>, as of 2019 threaded decompression hasn’t been implemented yet. <ref>https://man.cx/xz</ref>. Number of threads can be less that defined is file is not big enough for threading with the given settings or if using more threads would exceed the memory usage limit. <ref>https://man.cx/xz</ref>
Line 6: Line 9:
  
 
== Options ==
 
== Options ==
−
:<code>--keep</code> Once the target file has been successfully closed, the source file is removed unless --keep was specified
+
:<code>[[--keep]]</code> Once the target file has been successfully closed, the source file is removed unless --keep was specified
 +
:<code>[[-T]]</code> [[multi-threaded]] compression
 +
:<code>[[xz --help]]</code>
  
 
== Commands ==
 
== Commands ==
 +
* <code>[[xz -t]] file_to_test.xz</code>
 +
* <code>[[xz -q]]</code>
 +
* <code>[[xz -T]]</code>
 +
* <code>[[xz -T0]]</code>
 +
* <code>[[xz -9 file_to_compress]]</code>
 
* <code>xz --info-memory</code>
 
* <code>xz --info-memory</code>
 
<pre>
 
<pre>
Line 18: Line 28:
  
 
* Use <code>--verbose</code> option twice to get extra information about compression, such us number of threads to use or memory required. It can impact xz performance.
 
* Use <code>--verbose</code> option twice to get extra information about compression, such us number of threads to use or memory required. It can impact xz performance.
−
<code>xz -T0 --verbose --verbose file_to_compress</code>
+
<code>xz [[-T0]] --verbose --verbose file_to_compress</code>
 
<pre>
 
<pre>
 
xz: Filter chain: --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
 
xz: Filter chain: --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
Line 27: Line 37:
 
</pre>
 
</pre>
  
 +
 +
xz file_already_compressed.xz
 +
xz: file_already_compressed.xz: File already has `.xz' suffix, skipping
 +
 +
Compress directory using <code>[[xz]]</code> [[multithread]] capabilities
 +
* <code>[[tar c]] your_directory | xz [[-T0]] -c > your_file_name.tar.xz</code>
 +
 +
Uncompress file:
 +
unxz file.xz
  
 
== Examples ==
 
== Examples ==
−
* Test integrity:<code>xz -tv linux-3.18.19.tar.xz</code>
+
* Test integrity: <code>[[xz -t]]v linux-3.18.19.tar.xz</code>
 
* Compress data from <code>[[stdin]]</code>: <code>echo "DATA TO COMPRESS" | xz --verbose > file.xz</code>
 
* Compress data from <code>[[stdin]]</code>: <code>echo "DATA TO COMPRESS" | xz --verbose > file.xz</code>
 
* Compress all files in directory and subdirectory except already compresed (<code>.xz</code>) using all CPUs (<code>-T0</code>):
 
* Compress all files in directory and subdirectory except already compresed (<code>.xz</code>) using all CPUs (<code>-T0</code>):
−
:<code> find . -type f -not -name \*.xz -exec xz -T0 --verbose \{\} \;</code>
+
:<code>[[find]] . -type f -not -name \*.xz -exec xz -T0 --verbose \{\} \;</code>
 +
 
 +
  xz file.txt
 +
  xz: file.txt.xz: File exists
 +
 
 +
== Security ==
 +
* [[CVE-2024-3094 Backdoor in xz]]
  
 
== Activities ==
 
== Activities ==
 
# Compress your data using gzip and xz and compare sizes
 
# Compress your data using gzip and xz and compare sizes
 
# Compare sizes of Linux Kernel source code depending of compression tools used: https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/
 
# Compare sizes of Linux Kernel source code depending of compression tools used: https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/
−
# Compress your file using single thread <code>xz</code> and compress same file using all your available threads <code>xz -T0</code>
+
# Compress your file using single thread <code>xz</code> and compress same file using all your available threads <code>[[xz -T0]]</code>
−
# Compress your file with <code>xz -T0</code> and verify that you are using all your threads with [[htop]]
+
# Compress your file with <code>xz -T0</code> and verify that you are using all your [[threads]] with [[htop]]
 +
 
 +
== Related ==
 +
* <code>[[tar -J]]</code>
 +
* <code>[[zstd]]</code>
 +
* <code>[[xz -q]]</code>
  
 
== See also ==
 
== See also ==
 
* {{xz}}
 
* {{xz}}
−
* <code>[[zstd]]</code><ref>https://en.wikipedia.org/wiki/Zstandard</ref> compression utility released in 2015 which includes parallel (multi-threaded) implementations of both compression and decompression
+
* {{zstd}}
−
* {{cp}}
+
* {{Data compression}}
−
* {{du}}
 
 
* [[Intel QAT]] (QuickAssist Technology): https://software.intel.com/en-us/articles/how-intel-quickassist-technology-accelerates-nfv-use-cases
 
* [[Intel QAT]] (QuickAssist Technology): https://software.intel.com/en-us/articles/how-intel-quickassist-technology-accelerates-nfv-use-cases
  
 
[[Category:compression]]
 
[[Category:compression]]

Latest revision as of 08:28, 1 April 2024

xz utils[1] (previously LZMA Utils) compress or decompress (unxz) .xz and .lzma files but xz compresses by default to xz format.

xz support multi-threaded compression (-T flag)[2] since 2014, version 5.2.0. [3], as of 2019 threaded decompression hasn’t been implemented yet. [4]. Number of threads can be less that defined is file is not big enough for threading with the given settings or if using more threads would exceed the memory usage limit. [5]

xz has replaced gzip in some uses case such as in Arch Linux packages because it decreases package size about 30% smaller depending on the data.

Options[edit]

--keep Once the target file has been successfully closed, the source file is removed unless --keep was specified
-T multi-threaded compression
xz --help

Commands[edit]

xz --info-memory
Total amount of physical memory (RAM):  515,654 MiB (540,702,060,544 B)
Memory usage limit for compression:     Disabled
Memory usage limit for decompression:   Disabled
  • Use --verbose option twice to get extra information about compression, such us number of threads to use or memory required. It can impact xz performance.

xz -T0 --verbose --verbose file_to_compress 

xz: Filter chain: --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz: Using up to 48 threads.
xz: 7,925 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 9 MiB of memory.
.../...



xz file_already_compressed.xz
xz: file_already_compressed.xz: File already has `.xz' suffix, skipping

Compress directory using xz multithread capabilities

  • tar c your_directory | xz -T0 -c > your_file_name.tar.xz

Uncompress file: 

unxz file.xz

Examples[edit]

  • Test integrity: xz -tv linux-3.18.19.tar.xz
  • Compress data from stdin: echo "DATA TO COMPRESS" | xz --verbose > file.xz
  • Compress all files in directory and subdirectory except already compresed (.xz) using all CPUs (-T0):
find . -type f -not -name \*.xz -exec xz -T0 --verbose \{\} \;
 xz file.txt
 xz: file.txt.xz: File exists

Security[edit]

Activities[edit]

  1. Compress your data using gzip and xz and compare sizes
  2. Compare sizes of Linux Kernel source code depending of compression tools used: https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/
  3. Compress your file using single thread xz and compress same file using all your available threads xz -T0
  4. Compress your file with xz -T0 and verify that you are using all your threads with htop

Related[edit]

See also[edit]

  • ↑ https://man.cx/xz
  • ↑ https://man.cx/xz
  • ↑ https://git.tukaani.org/?p=xz.git;a=blob;f=NEWS;hb=HEAD
  • ↑ https://man.cx/xz
  • ↑ https://man.cx/xz
    • Retrieved from "https://www.wikieduonline.com/index.php?title=Xz&oldid=311230"

    Advertising: