How to fix Kafka Broker may not be available on 127.0.0.1 error?
May 8, 2023How to query data from Snowflake in Spark?
May 15, 2023This post describes the differences in sync and cp operations in S3 and which one should be preferred.
sync
aws s3 sync copies any files that have changed on the source location. It does not delete anything on the target location. Both source and destination may not look the same after this operation.
aws s3 sync –delete copies files changed in the source location. It deletes files from the target location that is not present. Both source and destination will look the same after this operation.
cp
aws s3 cp copies all files from source to target, regardless of where they are changed in source or not and it does not delete anything on the target.
sync should be preferred over cp because sync only works on the affected or changed objects rather than affecting all objects thereby has a cost advantage.
