[ art / civ / cult / cyb / diy / drg / feels / layer / lit / λ / q / r / sci / sec / tech / w / zzz ] archive provided by lainchan.jp

lainchan archive - /λ/ - 17682



File: 1469995802346.png (861.61 KB, 215x300, 1458627734166-0-λ.gif)

No.17682

Python 3 thread. What are you working on ?

  No.17701

Trying to learn Python3 but failing to find suitable packages that actually work :-)

  No.17702

File: 1470071025476-0.png (63.56 KB, 126x200, perl.png)

File: 1470071025476-1.png (66.39 KB, 142x200, ruby.png)

We don't need a thread for every language especially when superior scripting languages exists.

  No.17708

>>17702
node.js is the new Perl

  No.17710

We don't need pedos in every thread especially when a pedo board exists.
>>>/w/

On the topic of Python 3, get into the habit of using type annotations in function definitions. It helps a lot with larger codebases and with code you haven't touched in a while.

https://docs.python.org/3/whatsnew/3.5.html#whatsnew-pep-484
https://docs.python.org/3/library/typing.html#module-typing
https://www.python.org/dev/peps/pep-0484/

  No.17711

>>17710
Genuine question: what's the advantage of using Python with type annotations over... just using a statically-typed language?

  No.17712

>>17711
You keep whatever other advantages you perceived Python to have over $LANGUAGE. You have optional enforcement of types by an external tool, the interpreter doesn't enforce them. You can use as much or as little typing as you feel is appropriate for a particular project. It's the equivalent of placing your project wherever you please on a typing spectrum rather than being forced into the narrow band where $LANGUAGE lives.

  No.17714

>>17711
>>17712
Also types are arbitrary. You can use the type "degrees kelvin" with python type annotations if you want, you would have to add a comment in statically typed languages

  No.17718

>>17712
Nice answer,
typing spectrum mobility seems like a really useful feature for a language.

As for the other cruft on the thread, there really is no need for language elitism in this board. :^)
Thank you for your views, guys, but if you like another language better, then why don't you make a thread?

  No.17719

>>17714
You can have "degrees kelvin" as a type in pretty much any statically-typed language, too.

  No.17742

It's not exactly python but has anyone tried the programming language "coconut"? It's a language built on top of python which extends its functional features.

  No.17761

File: 1470279054397.png (14.7 KB, 200x105, a9c33e5.jpg)

I'm working on some quick Web crawlers and a brute force program. It's a really fun language. Pic related how I feel with people complaining python being dumbed down

  No.17764

>>17761
>people complaining python being dumbed down

It's funny because it's not dumbed down, decorators, generators, and funky metaclass magic is pretty advanced stuff. Unless they're just /g/-tier C drone wannabes who kvetch about garbage collection or significant whitespace.

  No.17822

Python 3 won't install on my Windows 8 machine. Something to do with redistributable stuff. A lot of things are giving me that error.

Anyway, can you guys explain how to create modern Python 3 modules/libraries? I simply can't find a straight forward tutorial that covers that topic. Python documentation seems to be either good enough or completely absent.

  No.17826


  No.17845

I'm having problems where I'm trying to search using the re resource but i need to remove part of the search itself to get to the final product. something like harambe.com/dicks/out but i need to remove the harambe.com part any suggestions?

  No.17847

>>17845
>>> import re
>>> data = "some text which includes harambe.com/clicks/out and some other stuff"
>>> expr = r"harambe[.]com(\S+)"
>>> mo = re.search (expr, data)
>>> mo is None
False
>>> mo.group (1)
'/clicks/out'
>>>
https://docs.python.org/3/library/re.html#re.match.group

  No.17852

Learning Python coming from C, Im having a blast so far. I working up to web scraping and making simple games with pygame.

  No.17863

>>17852
Yeah I'm learning scraping as well. A friend went missing a long time ago, and I'm going to find him with code.

  No.17865

>>17863
>A friend went missing a long time ago, and I'm going to find him with code.

Just stay away from apples, Alan.

  No.17866

>>17719
Not without defining a class or a type I don't think

  No.17885

>>17863
This could be one of the most beautiful story I've ever heard. Hope you'll find him.

  No.17890

>>17885
Thanks man. Me too. He was my best friend in school. None of my old friends know where he went. He left a mysterious text to one of my friends saying 'Don't listen to the rumors about me, they're not true'.

  No.17891

>>17890
What did the rumors say?

  No.17903

>>17891
None of us had heard any rumors. It was creepy actually. Nobody's heard from him since.

  No.17904

>>17903
>>17863

He has been trapped in the wired ;-) can you make a github repo of your code ? It may be funny

  No.17932

>>17904
Yeah I will do when the project has been developed a little more.

  No.18134

>>17826
Thanks! This was actually quite helpful. Especially:

https://packaging.python.org/distributing/#working-in-development-mode

I'm developing projects A, B and C, where:

>A is the main project

>B and C are dependencies of A

With requirements files I can link them locally and develop them all at the same time.

  No.18147

I write a lot of personal scripts in Python. Unfortunately my key-binder required some 2.7 only libraries, feelsbadman

  No.18193

Any recommended books specifically for Python?

  No.18195

>>18193
Think Python - How To Think Like A Computer Scientist by Allan Downey is a good intro to programming in general imo. you can get it here: http://greenteapress.com/wp/think-python-2e/

Zelle's "Python Programming: An Introduction To Computer Science" is another good book similar to that last one. Look for the 2nd edition which uses Python 3.

for a comprehensive but far lengthier introduction to the language, check out Mark Lutz's "Learning Python"

if you just want to dive in and do stuff then Sweigart's "Automate The Boring Stuff With Python" (https://automatetheboringstuff.com/) is definitely worth a look. from the same publisher (No Starch Press) Matthe's "Python Crash Course" is a good middle ground between a textbook proper and project based learning like the Sweigart book. I have taught children/teens with those two previous books so they are very easy to follow.

  No.18200


  No.18230

>>17845
if its actually a url just split('/')

  No.18231

File: 1472248018984.png (781.82 KB, 141x141, mywebsite.webm)

>>17682
Been using Flask to write a small dynamic portfolio. Honestly, flask has been such a breeze to use that it'll probably be my "go to" framework for any small sites I need to write. How is it looking so far?

  No.18264

>>18231
astoundingly generic

  No.18265

>>18264
Any suggestions on how to differentiate it?

  No.18266

>>18265
Hamburger menus are a UX nightmare, please don't use them.

  No.18267

>>18265
might be useful; a list I had made for designing a blog/notes page. ripped directly from a github readme. Goal was to look decent without any fancy anything.


## ``Rules''
http://webtypography.net/2.1.6

## ``Looks decent''
http://xxyxyz.org/bridges/ *Diagrams, Code, Links, Body*

https://edwardtufte.github.io/tufte-css/ *Margins, Line-Spacing*

http://www.nongnu.org/rdiff-backup/ *Background color*

http://www.gwern.net/Links#shu-thing-monadius *General ``Feel'', footnotes, navigation*

https://thesquareplanet.com/blog/students-guide-to-raft/ *Links, inter-text code, emphasis*

http://gentoo.no-ip.org/~/Anime/ ~~apache index~~

http://chimera.labs.oreilly.com/books/1234000001802/ch07.html *Diagrams*

https://www.w3.org/TR/css-text-3/#intro *2-Column Tables*

http://blog.zorinaq.com/release-of-hablog-and-new-design/? *Comments & highlighting*

### particularly for code
http://unitscale.com/mb/technique/pollen.rkt.html

http://xxyxyz.org/bridges/

http://galileo.phys.virginia.edu/classes/551.jvn.fall01/primer.htm

### books
Concrete Mathematics *Spacing, Body Font, Left-Margin Notes*

  No.18521

Automate The Boring Stuff
we talk about automating the Boring stuff with Python (3.x).

Here are a few resources

https://automatetheboringstuff.com/
https://gist.github.com/classam/9e07a36aa63624ca2dda75a1367a53c6

I actually created a little script based on selenium to automate things while browsing (logging in, uploading on multiple sites...) and i'm working on a script to sort and search my thousands of thousands of files.

I also wrote a lyrics finder for the command line, which I think is more efficient than going on websites with dozens of ads and praying for the lyrics to be here.

And you ?

Please share code. I will post if someone interested.

  No.18593

>>18231
What are you going to include in your portfolio?

  No.18597

>>18521
>I will post if someone interested.
Please do!

  No.18599

File: 1473527998819-0.png (495.43 KB, 200x125, new-window-manager-2.png)

File: 1473527998819-1.png (414.36 KB, 200x125, new-window-manager-1.png)

I want to make a terminal multiplexer like tmux but prettier in some aspects, then I'll make some terminal programs (I think I'll use termbox for the UI) following some guidelines I'm setting right now so that they all behave similarly and make for a good intuitive and user friendly terminal "ecosystem" or "suite" to use a couple of buzzwords. I think the project's path would look like this:
1. Multiplexer.
2. A shell so you can use other terminal software like vim, while keeping said guidelines when using the CLI.
3. Misc software like a music player, RSS client, etc. again following that set of guidelines.

I'll be doing it in python 3.5 because it'll be FOSS if I publish it, and I want it to be easy for other people to contribute as it's a very modular project.
This is my first big project and due to its nature its parts must piece together elegantly so it doesn't become a beast as it develops. That's the stage I'm in now, planning the overall structure.

In the end I hope it'll be at least as comfortable an environment as doing everything inside Emacs (like the guy in this pic), having keyboard-focused interfaces that are intuitive across programs but without the problems that come from Emacs' complexity (in my experience).

This is also an experiment to see if nice terminal software + w3m + mplayer can replace X (except flash).

  No.18604

>>18597
Okay. Tomorrow.

  No.18611

>>18599
Do you have a github I can follow this project on? I've always dreemead of replacing X for beautiful cli programs.

  No.18612

>>18611
Not yet, I can post updates here in the meantime. I'm getting started with making a text editor as a warmup, reading programming books without a project to practice is almost useless for me, so I'll be re-learning a lot at the beginning.

Anyway I think I'd use bitbucket, github's ToS suck.

  No.18613

>>18612
You are making a cli text editor in python 3.5? How?

  No.18615

>>18613
>>18613
Not technically CLI, I think the word is TUI like in ncurses. I haven't started yet, but my idea is using termbox to divide the screen in two, an upper part for the editing and a couple lines at the bottom for a status bar. Typing would just write, and the ctrl key would be used for shortcuts to edition actions like in vim's normal mode. For example, ctrl+dw would delete until the end of the word like vim's <esc>dw. Release ctrl and you're back to normal writing.

Is that what you meant to ask?

  No.18639

File: 1473574225761.png (74.37 KB, 160x200, cover.png)

finishing this soon, I'd like to ask for some more books to keep learning python 3 with if anybody knows

  No.18920

>>18639
https://shop.oreilly.com/product/0636920032519.do

Fluent Python is god tier for extending Python knowledge. I would say not every section of it is vital or things you need to know, but it goes well in depth and clearly explains a lot of intermediate or advanced Python stuff.

For example one chapter discusses something as simple as comparing "==" to "is" and how each one should be used and certain situations, etc.

Overall, an extremely useful book.

  No.18925


>>18639
Never read that particular book, just skimmed through it. I did read make your own games with Python and automate the boring stuff with Python by the same author. Automate the boring stuff was really good, lots of fun projects. You can read it free on the author's site.

  No.19008

File: 1475306252498.png (65.4 KB, 200x124, Screenshot - 09302016 - 11:13:25 PM.png)

im working on ....halp

  No.19009

>>19008
>muh fgets

  No.19013

>>19008
#include <errno.h>?

  No.19016

>>19013
>../Downloads/Python.../
I'm guessing this poster is building python so it isn't his code.
The error message is very strange for what otherwise should be a working project though.
Did you check if there was a ./config file to run first, or maybe a 'make config' step? or something along that line?
I'm guessing here that some #includes are added/omited depending on the OS

  No.19029

I started trying to rewrite a python program, I wanted to contribute to the project at first but the source code was so bad and full of useless stuff that I found it better to just replace all the parts with cleaner code incrementally.
It's going pretty well so far, but the more I use Python, the more I wonder why people praise it so much. It's often anything but clear, readable and obvious. Coming from Ruby, I really don't see any of its famed beauty and regularity.
I started rewriting parts of it in in Hy. It doesn't stray much from regular Python, but the macros definitely make it worth it for me, they let me "fix" many things that bug me about the language. I think it's a very nice thing to have, and though I wouldn't use Python or Hy by themselves, I'm really glad to have the option when I have to interact with Python codebases/libraries.
Anyway, have you tried Hy? What do you think of it? Would you want to see this kind of Lisp interface hackery in other languages? I sure would.

  No.19030

>>19029
A lot of languages in general would really benefit from a user-aimed interface to the AST.

  No.19051

Kind of a new programmer, currently writing something to randomly generate a haiku based off of inputs

  No.19140


  No.19141

>>19029
I've barely ever used any python but I am interested in why it has come to be bothersome for you, if only to be informed, care to elaborate pls?
Anyway, I do remember trying out Hy, I believe it is a great idea if only because of the macro flexibility.
Even though I do enjoy C, of course I often end up having more trouble than I should due to it's rigidity and it's dumb macro system. Often I would like to have some data structures be generated at compile time, as well as make some syntactic sugar over repetitive patterns in the low level stuff.
I actually want to create a low-level language that has the domain of C with the syntax of Scheme/Lisp. Such a language could thereafter be used to implement higher level lispus and it should be tightly optimizable as well...

  No.19142

>>19029
>It's often anything but clear, readable and obvious.
I kind of agree with this; I feel like it depends on what you're doing. I'm playing around with Tkinter to make a frontend for someone's script and it's kind of a mess; I can't just make a bunch of new lines between chunks of python because that's unreadable in its own right. However, I wrote a GUI automation script and it reads fine.

  No.22043

File: 1487022719164.png (496.87 KB, 200x134, Screen Shot 2017-02-13 at 1.46.54 PM.png)

This is part of a much larger project, but currently trying to get a script running that will retrieve the list of my friends from fb.

I'm using "urllib2" (and using an HTTP request to get the URL from my fb account) and BeautifulSoup to retrieve the HTML from the page, but I'm running into a small problem: There is a lot of data and I don't think my Friends' names are on there.

(And to clarify, I've tried using the Facebook API, but there's something funky with that. I won't be able to get the list of my friends without the proper permissions (my friends would have to be using my app for me to include them. All I want is a list of my friends) )

Here's what I have so far:
'
from bs4 import BeautifulSoup
import urllib2
response = urllib2.urlopen("")
soup = BeautifulSoup(response, 'html.parser')

print(soup.prettify())

  No.22052

>>19141
>care to elaborate pls?
It makes useless distinctions between statements and functions. For example, you can't return from an "if" unless it's an inline if, which is dumb because if statements DO return (you can clearly see this in the REPL) and in the AST there is an "if" expression (which is the one Hy uses by default), but somehow trying to assign its result is a syntax error because it's a statement, which is compiler speak for "because I said so".
In fact, I think the forced indentation just gets in the way of writing code, because there are constructs which may sometimes require a weird indentation (for example lambdas, which STILL don't allow statements inside of them).
For some reason, Python users look afraid of functional idioms, despite using them a lot. For example, lots of Python users seem to think list comprehensions are the best thing after sliced bread. I'm not sure they realize it's just (bad) syntactic sugar for a simple map().
The language is ultimately uncomposable and pervasively procedural, due to its syntax and idioms. It's not that bad, but I wouldn't recommend using or learning it.
If you want a batteries-included scripting language, you should try Ruby; it's not perfect, but it has much more solid functional primitives, can heavily rely on metaprogramming, and it's flexible enough to accomodate many different programming styles.

>a low-level language that has the domain of C with the syntax of Scheme/Lisp.

I think you're searching for Common Lisp. It is actually a very procedural systems language, despite its reputation, and it's very tightly optimizable. You can optimize important functions iteratively very easily in the REPL by inspecting the compiled code.

  No.22054

>>22043
I don't think you can do that with just beautifulsoup, you probably need a lot more, possibly a headless browser.
Facebook uses React to load its content, and probably a fuarrrkload of code written specifically to prevent what you're trying to do there.
They probably don't offer a friends list anymore because such an API violates privacy and could easily be used to spam people. Most people are retarded or ignorant enough to dismiss warning or errors and then blame Facebook for their own incompetence afterwards, so even if Facebook put a huge warning in red characters saying that by accepting this you acknowledge that you trust the app not to make improper use of your friends list etc etc, people would still not read and then complain to Facebook when everybody in their friends list starts getting spammed.

  No.22055

>>22054
I agree, scraping a site like facebook is a task from hell. Unless you really want to spend a long time on such a project I would say you should probably forget about it.

I've used BS only for scraping Russian oil executive information and basic sites like phrack and textfiles. The more text based the sites, the better it is for scraping. Phrack was very easy since you can just filter out anything that isn't in a <pre> tag. I wish other sites would be that simple.

Also, if you are interested in headless browsers check out phantomjs.

  No.22060

>>22052
You may be right. The language itself has its problems. And often enough I think I am writing "unpythonic" code when using map, filter, reduce… too much.

But I can not get away from it because it has such nice libraries. NumPy/SciPy alone are great and there is the large standard library which up to now never disappointed me. For me it's the number one language for getting things done™. Only if things get more complex as initially expected it gets ugly fast.

  No.22061

>>22054
>>22055

Alright... Thanks Lainanons...
It's kind of unfortunate that I won't be able to implement this: https://arxiv.org/pdf/1310.6753v1.pdf

What I might end up doing is making a sort of a "graph generator" app and from there try to do what I need.

Once I have that, maybe I can try to get in contact with some FB developers to let me implement this (Because the API does have a "taggable_friends" parameter that might be of use, but I would need to get my app approved)?


This is more of a "let's see if I can do this" and a resume builder more than anything, so I'm not too worried I won't be able to use FB's actual data.

Small question, though, if I can't parse it using BS, how would a headless browser like PhantomJS help me? (I'm new to the 'headless browsers' scene, and so I'm not exactly sure what they can be used for that can't be achieved by scripting languages like Python)

Thanks again for the help!

  No.22074

>>22060
I agree it has lots of good libraries, so I'm glad there's stuff like Hy or Coconut for when I want to use them.

>>22061
I've never used a headless browser myself, but I've heard of a guy using one to scrape his facebook feed and it sounded like hell. They can do whatever a normal browser can do, if you program them well enough.
I just found the article I was thinking about, this should hopefully dissuade you from pursuing this:
https://franciskim.co/dont-need-no-stinking-api-web-scraping-2016-beyond/

  No.22090

>>22074
Coconut seems really nice. Think I'll try it out in my next project. Especially the Jupyter support sold me.

  No.22231


  No.22267

>>17708
Inferior Regex standard, Javascript can't do negative lookaheads or behinds.

>>17764
I dislike forced whitespace, but why do people dislike garbage collection? Is it the way it's made, or the concept itself?

I'm doing a SSH-based cloud storage implementation with no databases or other permanent state (except for the filesystem metadata obviously). I posted about it on /lambda/ about a month ago, but didn't get much in the way of help.

  No.22271

I'm working on a restful API using the Flask framework. API is backend for a mobile app prototype.

  No.22353

File: 1488445800071.png (1.14 MB, 200x200, result11.png)

Working with Pillow on a script that randomly generate images from a set of other images in a media folder, and then post it on a Facebook page. Later I want it to directly grab image from other Facebook pages, maybe mix it with the local folders, and do the same thing.

I'm getting kind of stuck on how I should organize this program : In Pillow there is functions that only need one image object to work on, and others requiring two, even three if a mask is involved. For now I have a set of function used to randomly build a list of files, either tuples or single elements, check the type of the list element and apply one of the adequate functions.

There is no OOP (for Python I prefer to use OOP) in there, and it feels a little wonky for me, as the complexity of this script will grow quickly and it will become difficult to extend later.

It's very funny to work on though.

  No.22378

I'm trying to lean PyQt, really dislike it honestly.
Right know I am having problem with a widget:

I have mainwindow class with a Qstackedwidget as central widget. Then other classes for the widgets I put there.
I would like to access a mainwindow's function from some of the child widgets so, in the widget, I do something like "self.parent().mainwinfunction()". Problem is that the parent is the "Qstackwidget" object not the mainwindow, so I have to call parent() two times.
The result looks like this:

  class MyMainWin(QMainWindow):
def __init__(self):
super(MyMainWin, self).__init__()
self.CentralWidget = QStackedWidget()
self.setCentralWidget(self.CentralWidget)
self.my_widget = MyWidget(self)
self.CentralWidget.addWidget(self.my_widget)

def hello(self):
...

class MyWidget(QWidget):
def __init__(self):
super(MyWidget, self).__init__(parent)

def something_happen(self)
self.parent().parent().hello()

am I doing it right? is there a better/cleaner/morecorrect solution? would a custom signal be better?

  No.22382

File: 1488663055435.png (1.46 MB, 200x93, Screen Shot 2017-03-04 at 1.13.45 PM.png)

Trying to make a script to steal money from starving artists

In bandcamp, you can go around paywalls by clicking "Inspect element" on the Play button for a song, seeing the largest file (or an mpeg file that shows up once you press the button), and click "Open in other New Tab"

My question is, given the URL for a song, how would I simulate this? I would need the location of the mpeg file (and I could easily download from there), but I'm not sure how to go about actually finding the "request URL". I'm thinking of doing a button press and checking for resources, but that might not be what I need...

This is more of a practicing-web-scraping thing more than anything, so I probably won't do anything nefarious.

Thoughts?

  No.22385

>>22382
Selenium can manipulate the site to get the URL.
Scrapy can allow you to do this on a large scale https://doc.scrapy.org/en/latest/topics/media-pipeline.html

I've been using scrapy at work for 10 months now or something, and it's very impressive what you can do with it.

Actually, I don't know a lot about Bandcamp, but if all bandcamp sites are laid out the same, you don't even need Selenium. You can just Scrapy, and tell Scrapy to click the button and grab the files. If you figure that out, you can actually write a scrapy script to do it for all your favorite bands at once

  No.22393

File: 1488742573079.png (192.28 KB, 200x175, YamaskPacha.png)

>>22385
Thanks a lot, my pythonic brethren!

I do have a small question, once the button is pressed, a request for the mpeg file is sent. How would I 'intercept' that? Better yet, how would I find out what http request have been sent after the button is pressed?

Once again, thank you very much for pointing me in the right direction. I appreciate it!

  No.22394

>>22393
Can you give me an example of a bandcamp site with a paywall so I can try what you are saying to get around it. It would give me more insight, I went to the one you had in the SS, but it didn't have a paywall to listen.

Do you mean paywall to download? To be honest I haven't looked at it for over ten minutes but here is where I would start. The idea is that it is totally doable, because anything that works on your browser is scrapable.

Here is the xpath for the button
id('trackInfoInner')/x:div[1]/x:table/x:tbody/x:tr[1]/x:td[1]/x:a/x:div

I would try to curl the url of the music with -H accept audio, and if it works then you know you just need to write a method to click on all the play buttons of all the songs. Scrapy is unique in that it can do that simultaneously, but just write test cases.

Though since it is streaming, you might need to write the scrape to make a request first, and then grab everything in the call back. Something like this:

def parse(self, response):
url = response.xpath('//a[contains(@href,".interestingfileextension")]/@href').extract_first()
return scrapy.Request(url, callback=self.parse_file)

def parse_file(self, response):
# response here is the contents of the file
yield response.body

  No.22395

File: 1488752139883.png (7.77 MB, 200x125, Screen Shot 2016-02-10 at 1.18.23 PM.png)

>>22394
>Can you give me an example of a bandcamp site with a paywall
Sure, though I may have misspoken with my use of the word "paywall". What I mean is that the albums themselves cost money and it is possible to download the music files (so yes, paywall to download) in the method I described earlier (click button, check network statistics, click "open in new tab" for any mpeg audio)


In the link below, when you click on the play buttons, you send/receive a request for an mpeg file.
https://midwestcollective.bandcamp.com/album/before-the-night

I have a couple of questions: 1) How would you get the Xpath for a button besides using the "Inspect Element" button, and 2) Once I make the request (for the mpeg file) by clicking the button, how would I know which is the correct HTTP request?

also, can I use requests instead of curl?
Thank you, lainanons!

  No.22399

>>22395
Sorry I mispoke. cUrl was just a starting point. So if you are using Scrapy, the scraping frame work-- the xpath for the button would be where you make the request. You know what the correct http request is by using inspect element, and checking the script or network tab and seeing which one is making the call to stream the music. It's much simpler then it sounds, you can start by just dicking around in firebug, or the default element inspector thing, and looking for how the request is done.

  No.22401

>>22395
  $ wget --user-agent= -O step0 https://midwestcollective.bandcamp.com/album/before-the-night
$ grep -E -e '"mp3-[[:digit:]]+":"[^"]+"' -o step0 > step1
$ sed -r -e 's/"mp3-[[:digit:]]+":"([^"]+)"/https:\1/' step1 > step2
$ cat step2
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=6d1730f194b8faf6f87030f01b3bd1ca&id=3494315798&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=36511d114eea6a093566c68ba3aa9a79&id=3854515479&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=0b3dabb9c0f2013a6dd617b2bbffeff6&id=1325625993&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=edf445a51c0feb1768e948373bc35da0&id=1453081794&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=c2dc0e84d8b7fde6efabd8367a3c4aa5&id=703059047&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=efb33ee55ee31fc90a7dec061176f101&id=2158379178&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=4fe52d0598f4444aae79f5c4fabf7857&id=1235679296&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=c3f53abcd2f8926671f53410b419a92b&id=1644828361&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=8556b711141ebd0c9adf8f4be792557a&id=3063806170&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=454458fc9a7b62c9fa70808aedbcd5b9&id=1130823002&stream=1&ts=1488179514.0
https://popplers5.bandcamp.com/download/track?enc=mp3-128&fsig=206608966c610642c07fb5f70806a9ee&id=2777707391&stream=1&ts=1488179514.0

  No.22403

File: 1488761657189.png (3.86 MB, 200x144, new.png)

I wrote a python script to merge images

  No.22404

>>22403
That's really cool! Did you use any particular libraries or did you use the file's binary low-level representation (or both)?

>>22399
Alrighty... I have a much better idea of what to do now
>Go to website
>Use scrapy to locate the button (which conveniently has the html tag "playbutton")...
Not entirely sure what I'd use for this, more generally, does Scrapy have a neat way of returning an xpath if you know the html layout, or do you need to figure it out?
>Press the button (?)
>Use scrapy's inspect element functionality to check network requests.
>search for ".mpeg" files
>download

Honestly, I think the hardest part for me would be to use the xpath of the button. Once I know the xpath for the button, would I be able to "press" it?
Also, when you say firebug, that's the same as the thing that pops up with "inspect element" right? How would I use firebug with my script?

Thanks so much for helping, btw. I appreciate it.

>>22401
Sed is black magic and there's nothing you can say to convince me otherwise.
What do your commands do?

  No.22405

File: 1488778192436.png (155.42 KB, 132x200, unix-magic.jpg)

>>22404
Not that lainon, but yes sed certainly is magic.
In their command the "-r" enables extended regexp support, the "-e" is not needed in this case but allows you to string multiple sed scripts together without calling the sed binary multiple times, the "s" means the start of a substitute command (structured in the form 's/match/replace/'). In this script "/" are used as delimiters and the rest I am going to simplify because that is way more complicated then it needs to be.
  sed 's_^.*\(//.*\)_https:\1_' step1 > step2 
For the sake of readability I've changed the delimiters to "_" this means I don't have to escape the "//". Sed doesn't care either way.
In my script sed matches anything from the beginning of the stream (represented by "^") and any characters (represented by the ".") any number of items (represented by the "*") up until the first "//". Then I have marked the "//" and any characters after it ".*" by enclosing them in brackets. Now normally a bracket would just match a bracket but by escaping it with a "\" it tells sed to store everything inside in memory. The next "_" ends the "match" section and begins the "replace" section. We are replacing everything that was matched with "https:" and then using the "\1" we tell sed to place the first stored pattern after the "https:" (yes this means sed can store up to 9 patterns per sed script".

Now since I really don't expect you to understand the yammering above I will just link you the resource I used to learn pretty much everything I know about sed: http://grymoire.com/Unix/Sed.html

  No.22406

>>22405
>'s_^.*\(//.*\)_https:\1_'
>up until the first "//"
>the first
  $ echo 'a//b//c' | sed 's_^.*\(//.*\)_https:\1_'
https://c
>pretty much everything I know about sed
>

  No.22410

File: 1488841214637.png (1.71 MB, 200x150, lain.gif)

>>22404
I used the Image module from PIL

  No.22416

File: 1488853263374.png (200.83 KB, 200x143, 1439417928625.jpg)

>Python 3 thread. What are you working on ?
Using Common Lisp more since I don't want to relearn Python again.

  No.22425

After finishing codecademy course and learning python the hard way I made a Caesar chiper encryption/decryption tool and I'm planning on making a roman/Arabic numbers translator.
What should I do next to improve my learning? I want to start some serious project but can't seem to find one

  No.22430

>>22425
think of software you need or simple software you like, and then go read it's source and make it better/different.

So for instance if you really like Mutt, try to make your own version of Mutt in Python. When I was starting out with Python I made a file manager and it taught me a lot.

  No.22480

I'm new to programming and I wanted to know roughly how long it'd take before I'd know how to make things that are actually useful or interesting like markov chains?

I mostly want to use programming to automate things (I started reading Automate the Boring Stuff with Python last night) but I'm also interested in the more "science" side of CS, like AI, which I've heard Python, much like Lisp, is good for.

  No.22552


  No.22741


  No.22749

>>22741
I like this idea a lot. Girlfriend and I were actually thinking of doing something similar. Keep up the good work.