What is the best utility/library/strategy with Python to copy files across multiple computers? -

-

i have data across several computers stored in folders. many of folders contain 40-100 g of files of size 500 k 125 mb. there 4 tb of files need archive, , build unfied meta data system depending on meta data stored in each computer.

all systems run linux, , want use python. best way copy files, , archive it.

we have programs analyze files, , fill meta data tables , running in python. need figure out way copy files wuthout data loss,and ensure files have been copied successfully.

we have considered using rsync , unison use subprocess.popen run them off, sync utilities. these copy once, copy properly. once files copied users move new storage system.

my worries 1) when files copied there should not corruption 2) file copying must efficient though no speed expectations there. lan 10/100 ports being gigabit.

is there scripts can incorporated, or suggestions. computers have ssh-keygen enabled can passwordless connection.

the directory structures maintained on new server, similar of old computers.

i @ python fabric library. library streamlining use of ssh, , if concerned data integrity use sha1 or other hash algorithm creating fingerprint each file before transfer , compare fingerprint values generated @ initial , final destinations. of done using fabric.


Comments

Post a Comment

Popular posts from this blog

jasper reports - Fixed header in Excel using JasperReports -

-
how can create excel report fixed header using japserreports? mean need header fixed when scroll excel file. this possible adding couple properties in jrxml file each report. take @ advanced excel features freeze panes. if wanted freeze after first column header (down left side basically) this: <statictext> <reportelement style="sans_bold" mode="opaque" x="0" y="60" width="104" height="20" forecolor="#ffffff" backcolor="#666666"> <property name="net.sf.jasperreports.export.xls.auto.filter" value="start"/> <property name="net.sf.jasperreports.export.xls.column.width" value="110"/> <property name="net.sf.jasperreports.export.xls.freeze.column.edge" value="left"/> </reportelement> &
Read more

media player - Android: mediaplayer went away with unhandled events -

-
i need duration of audio file series of voice announcements need play app. have added audio files resources , play fine. sample code below works perfect intended purpose: return duration of audio files. here code: float getdurationofaudioresource(locationenum loc, context context){ float duration = 0; try { mediaplayer mp; mp = mediaplayer.create(context, getaudioresource(loc)); duration = mp.getduration(); mp.release(); mp = null; } catch (illegalstateexception e) {e.printstacktrace(); logerror(25, "testdescitem:fault::could not open mediaplayer object audio resource.");} return duration; } here's weird thing. code called in main activity prepares set of audio instructions given test. there no errors within activity. second activity called, long string of errors on logcat. 03-07 13:23:43.820: i/actionlogger(21435): gentest_info_test #0 created. 03-07 13:23:43.830: i/actionlogger(21435): gente
Read more

python - ('The SQL contains 0 parameter markers, but 50 parameters were supplied', 'HY000') or TypeError: 'tuple' object is not callable -

-
import pyodbc,nltk,array cnxn = pyodbc.connect('driver={mysql odbc 5.1 driver};server=127.0.0.1;port=3306;database=information_schema;user=root; password=1234;option=3;') cursor = cnxn.cursor() cursor.execute("use collegedatabase ;") cursor.execute("select * sampledata ; ") cnxn.commit() s=[] j=[] x=[] words = [] w = [] sfq = [] pos=[] entry in cursor: s.append(entry.injury_type),j.append(entry.injury_desc) nltk.tokenize import punktwordtokenizer nltk.corpus import stopwords tokenizer = punktwordtokenizer() english_stops = set(stopwords.words('english')) in range(0,26): # filter stop words tokenizer.tokenize(j[i]) w.append([word word in tokenizer.tokenize(j[i]) if word not in english_stops]) in range(0 , 26):#converting tokenzied text ito string sfq.append(" ".join(w[a])) replacers import regexpreplacer replacer = regexpreplacer() in range (0,26):#pos tagging replacer.repl
Read more