haskell - Using pipes-csv to parse Latin-1 encoded content? -


i'd use pipes-csv parse large csv files, turns out these csv files latin-1 encoded , turns out pipes-csv, , cassava library depends on, assume utf-8. ends producing parsing errors need handle.

the way i've approached duplicate records hold csv data text fields bytestring fields in dup. decode dup, manually translate latin-1 strings utf-8 , create final record. inelegant least.

is there better way?

per daniel's suggestion, here have far:

import qualified pipes.text.encoding pte import qualified pipes.bytestring pb  withfile "file.csv" readmode $ \h ->   let transcode = pte.decodeiso8859_1 . pb.fromhandle ~> pte.encodeutf8   runeffect $ decodebyname (void . transcode $ h) >-> process

it trades off unnecessary records unnecessary re-encoding of text, it's improvement. don't suppose there way without doing either of these unnecessary things?


Comments

Post a Comment

Popular posts from this blog

routing - AngularJS State management ->load multiple states in one page -

Image
i developing site in angularjs ui-router. have situation on button click need change , load 3 state page , 3 looks image .please me how can load multiple pages. in 1 page you need configure router config this: $stateprovider .state("state1", { url: "/url1", controller: "firstcontroller", views: { view1: { templateurl: "templates/template1.html" }, view2: { templateurl: "templates/template2.html" }, view3: { templateurl: "templates/template3.html" } } }).state("state2", { url: "/url2", controller: "secondcontroller", views: { view1: { templateurl: "templates/template4.html" }, view2: { templateurl: "templates/template5.html" }, view3: { templateurl: "templates/template6.html" } }).state("state3", { ...
Read more

python - GRASS parser() error -

i'm trying use grass on python 2.7, i'm having problems @ setting script on idle, i'm getting error @ parser() function: here script: import os import sys gisbase = os.environ['gisbase'] = 'c:\program files (x86)\grass gis 7.0.1rc1' gisrc = 'c:\grassdata' gisdbase = 'c:\grassdata' location = 'newlocation' mapset = 'tc' ld_library_path = 'c:\program files (x86)\grass gis 7.0.1rc1\lib' path = 'c:\program files (x86)\grass gis 7.0.1rc1\etc';'c:\program files (x86)\grass gis 7.0.1rc1\etc\python';'c:\program files (x86)\grass gis 7.0.1rc1\lib';'c:\program files (x86)\grass gis 7.0.1rc1\bin';'c:\python27';'c:\program files (x86)\grass gis 7.0.1rc1\python27';'c:\program files (x86)\grass gis 7.0.1rc1\msys' pythonlib = 'c:\python27' pythonpath = 'c:\program files (x86)\grass gis 7.0.1rc1\etc\python' sys.path.append(os.path.join(os.environ['...
Read more

Swift game error message -

i following ios game tutorial. cannot figure out wrong here. error on these 2 lines. let newangle = percent * 180 - 180 cannon.zrotation = cgfloat(newangle) * cgfloat(m_pi); 180.0 //error "use of unresolved identifier 'percent'" the tutorial uses: cannon.zrotation = cgfloat(newangle) * cgfloat(m_pi)/ 180.0 but changed error suggested import spritekit class gamescene: skscene { var cannon: skspritenode! var touchlocation:cgpoint = cgpointzero override func didmovetoview(view: skview) { /* setup scene here */ cannon = self.childnodewithname("cannon") as! skspritenode } override func touchesbegan(touches: set<uitouch>, withevent event: uievent?) { /* called when touch begins */ touchlocation = touches.first!.locationinnode(self) } override func touchesmoved(touches: set<uitouch>, withevent event: uievent?) { touchlocation = touches.first!.locationinn...
Read more