What happens to files with duplicate hashes in a processing set using Global deduplication?

In the scenario of global deduplication within a processing set, the correct function is to retain only unique files by identifying duplicates based on their hashes. When files have duplicate hashes, only the first version is designated as the valid instance and will be kept in the processing set. This ensures that storage is optimized by eliminating redundant data, yet it maintains the integrity of the original file.

Choosing to publish all versions of a file to the workspace would undermine the purpose of deduplication, which is to prevent unnecessary duplication and conserve storage resources. Thus, the process inherently keeps just one representative instance of each unique file rather than all versions. This practice aids in managing data effectively, enhancing retrieval speed, and reducing storage costs by retaining only the necessary components within the dataset.