Copy Multiple Files From S3 Bucket


Answer :

Also one can use the --recursive option, as described in the documentation for cp command. It will copy all objects under a specified prefix recursively.



Example:



aws s3 cp s3://folder1/folder2/folder3 . --recursive


will grab all files under folder1/folder2/folder3 and copy them to local directory.



There is a bash script which can read all the filenames from a file filename.txt. 



#!/bin/bash  
set -e  
while read line  
do  
  aws s3 cp s3://bucket-name/$line dest-path/  
done <filename.txt


You might want to use "sync" instead of "cp". The following will download/sync only the files with the ".txt" extension in your local folder:



aws s3 sync --exclude="*" --include="*.txt" s3://mybucket/mysubbucket .


Comments

Post a Comment

Popular posts from this blog

C Perror Example

Defined in header <stdio.h> void perror ( const char * s ) ; Prints a textual description of the error code currently stored in the system variable errno to stderr . The description is formed by concatenating the following components: the contents of the null-terminated byte string pointed to by s , followed by ": " (unless s is a null pointer or the character pointed to by s is the null character) implementation-defined error message string describing the error code stored in errno , followed by '\n' . The error message string is identical to the result of strerror ( errno ) . Parameters s - pointer to a null-terminated string with explanatory message Return value (none). Example # include <stdio.h> int main ( void ) { FILE * f = fopen ( "non_existent" , "r" ) ; if ( f == NULL ) { perror ( "fopen() failed" ) ; } else { fclose...
Read more

Converting A String To Int In Groovy

Answer : Use the toInteger() method to convert a String to an Integer , e.g. int value = "99".toInteger() An alternative, which avoids using a deprecated method (see below) is int value = "66" as Integer If you need to check whether the String can be converted before performing the conversion, use String number = "66" if (number.isInteger()) { int value = number as Integer } Deprecation Update In recent versions of Groovy one of the toInteger() methods has been deprecated. The following is taken from org.codehaus.groovy.runtime.StringGroovyMethods in Groovy 2.4.4 /** * Parse a CharSequence into an Integer * * @param self a CharSequence * @return an Integer * @since 1.8.2 */ public static Integer toInteger(CharSequence self) { return Integer.valueOf(self.toString().trim()); } /** * @deprecated Use the CharSequence version * @see #toInteger(CharSequence) */ @Deprecated public static Integer toInteger(String self) { return toInteg...
Read more

530 Valid Hostname Is Expected When Setting Up IIS 10 For Multiple Sites

Answer : Solution 1: When configured with two or more hostnames, the correct virtual host name and username must both be sent in the username by the ftp client. Separate the site name and user with the vertical line symbol: | www.example.com|MyUser So, in your FTP Client use this for the username: ftp.telefonievergelijken.nl|tv_ftp Solution 2: It appears that you're attempting to connect to the FTP site using a hostname which is not currently configured in any of the bindings to the FTP site within IIS. I base this only on the error output from Filezilla which you have included, as you have censored the hostname (even in example form) from the output, so there isn't much more to go on. You'll need to configure a binding on the FTP site which matches the hostname you are using to connect to the FTP site (whether that be from Filezilla or any other FTP client). EDIT: From your updated post information, I notice that your bindings for the FTP site are indeed in...
Read more