000webhost

Web hosting

Sunday, January 28, 2018

Net Neutrality or Google/YouTube Throttling Traffic?, Random Stuff, and More

- over the last few days I've noticed YouTube traffic slowing down noticeably. Not just in a particular area but over entire regions. Seems as though someone may be implementing mass scale throttling of traffic traffic? I decided to do a look at this further as it was creating trouble with my YouTube news downloader script...
youtube throttling traffic youtube-dl
traffic shaping google
- in the past I've often using proxies to get around such problems (but normally this has been due to local traffic shapers, firewalls, proxies, etc...)  but this time it doesn't actually seem to work?
- looked a bit deeper. Is not a manual counter-measure, something automated? Multiple layers we're dealing with here. Seems to be actively looking for multiple streams/jets/connections and/or bandwidth levels? Every once in a while things go back to normal (intermittent) though?
- I looked online. Frustrating thing about online is that people don't really often provide useful answers...
Throttle youtube-dl option #4550
Avoid bandwith throttling #6923
- watching traffic dumps is an option but it doesn't tell you things in real time which is often what you need. I've been using iptraf/wireshark to watch traffic speed as it crosses networks...
watch all urls currently going through linux cli
- looked at alternatives websites, plugins, and download options. Seem to be getting slowed down as well?
- without further investigation could be anywhere upstream though? Could be ISP level, could be NBN level, could be submarine cable level, could be YouTube/Google server level, etc... One thing that has been fascinating has been the relatively bursty nature of the traffic in question. I always thought it was much smoother then this?
- if you track their most recent behaviour it feels like Google is in 'cost cutting mode'? Some of the savings they seem to be targeting doesn't make any sense though and surely wouldn't make much of a difference? What's become very obvious to me is that Google has become just another corporate? It feels like many US organisations use the cover of 'liberation, democracy, innocence, etc...' as a marketing mechanism for whatever other activities they may be carrying out nowadays?
Mohan and Kincl said: "Though these changes will affect a significant number of channels, 99% of those affected were making less than US$100 per year in the last year, with 90% earning less than US$2.50 in the last month.

"Any of the channels who no longer meet this threshold will be paid what they’ve already earned based on our AdSense policies. After thoughtful consideration, we believe these are necessary compromises to protect our community."

In April last year, Google set a YouTube Partner Programme requirement of 10,000 views for videos to carry ads. This time, that has been cut further.

Thousands of contractors will review the clips on Google Preferred. Within that section, the company says it will have "a three-tier suitability system" that will “give them appropriate placements for their brand, while understanding potential reach trade-offs", 
- I've obviously considered the possibility of malware and otherwise but it doesn't seem possible given the environment that I'm working from...
- I obviously looked around and found out that US laws may have changed and everything has gone to crap globally? 
Net neutrality is the principle that Internet service providers must treat all data on the Internet the same, and not discriminate or charge differently by user, content, website, platform, application, type of attached equipment, or method of communication.[4] For instance, under these principles, internet service providers are unable to intentionally block, slow down or charge money for specific websites and online content.
The FCC just voted to gut net neutrality rules, letting Internet providers like Verizon and Comcast control what we can see and do online with new fees, throttling, and censorship. But we can still get Congress to stop this - by passing a "Resolution of Disapproval" to overturn the FCC vote.

We can win. Write and call Congress now! Learn more.
- out of curiosity I looked up net neutrality law changes? Feels possible this may be the problem?
Net neutrality is an important issue that can be difficult to fully understand. Thankfully, Burger King has stepped into the fray and produced a commercial that explains the concept using a metaphor we can all understand: the Whopper.

In an ad titled “Whopper Neutrality,” the fast food giant demonstrates just how stupid and unfair the internet may become when net neutrality is rolled back. In the ad, real customers approach the till and attempt to order burgers. Staff members explain that people who want their burgers served quickly will pay more than those who wait longer for their burgers. The burgers are sitting there ready to be served, they explain, but only those who pay a premium will get their burgers right away.
- only people who can genuinely confirm that this is the work of new 'net neutrality laws' are those that work at ISP's or those at major data centres. Huge problem for those who deal with 'big traffic'. Smallest and growing firms likely to be hit hardest? They can't really plan easily because traffic levels are 'bursty'. SME parts of tech sector likely to have problems? Build some stuff that measures correct traffic levels in past... A lot of politicians don't seem to 'get it' or else they're ignoring arguments they don't like? You can sort of tell that Google are struggling to maintain an eye on their environment at times. Lot's of old code still lying around that shouldn't?
- obvious solution is take things offline again? Build own infrastructure? Same methods to bypass restrictions in authoritarian countries now have use in supposed liberal democratic capitalist countries? Other methods include alternative websites, mimic traversal via 'better servers/lines', etc...
- opportunities abound for taking 'everything offline'? In reality, most of us only traverse a small amount of the Internet. Much of the Internet is also composed of 'garbage' that wastes a lot of people's time? What's funny about my situation is that I'm only really dealing with certain news sites. The news sites in question often have their own separate video download area which is quicker then dealing with YouTube?
- there aren't too many alternative options out there to be honest? Have been examining underlying code to see whether I can do something possibly different? A lot of people noticed the same thing that I have. Quick download at start which then gets throttled to about 50KB/s. This is fine for small files as there is no impact, larger files you can stop/start or else use multiple streams, etc?
video-dl script
get youtube download link videos
- I fiddled around with various external download managers since youtube-dl allows for aria2c,avconv,axel,curl,ffmpeg, httpie, and wget as alternate download managers. I kept on fiddling around with things  (and they worked in standard/fast transfer speeds for a while) but ultimately it resulted in errors with Google/YouTube services from time to time? Oddly shaping feels automated sometimes and at ther times not? Indicates they may not/don't have 100% Standard Operating Environment (SOE) going at the moment? Putting IP address of server in question into browser from iptraf results in Google server at other end. Feels almost eeery at times. Almost perfectly tracking 50KB/s Started a few days/a week ago? Seems to be everywhere (I and others have written scripts/programs to detect bandwidth of connection in question and likely requirements for traffic and automatically shapes traffic based on this. Wonder whether this is similar or a more brute force/less intelligent approach...)? Following key line (using aria2c) seems to regain more 'standard download' speeds (~200-300KB/s)... Note, "--max-connection-per-server" feels like a key parameter here. Don't go too high or else you start to get throttled/shaped...
../youtube-dl -f 36 --external-downloader aria2c --external-downloader-args "--max-connection-per-server=4 --min-split-size=1M" URL1 URL2 URL3
../youtube-dl -f 249 --external-downloader aria2c --external-downloader-args "-s 5 -x 4 --max-connection-per-server=4 --min-split-size=1M" URL1 URL2 URL3
../youtube-dl -f 36 --external-downloader httpie --external-downloader-args "-d" --proxy socks5://127.0.0.1:9150 URL
https://github.com/Code-Hex/pget
- you may have to revert to an alternate mechanism of encapsulating proxy capability with certain proxies if you're using aria2c
Support Socks proxies #153
https://github.com/aria2/aria2/issues/153
https://tor.stackexchange.com/questions/6868/does-traffic-still-go-through-tor-with-this-exception-error-in-aria2c
- as indicated by my above research my guess is that even if 'net neutrality' is rolled out across the board it may be possible to find ways around it. That said, you still have to find a way around it and be able to distribute the solution en-masse in public view prior to others implementing a counter-measure and so on and so forth?

Random Stuff:
- as usual thanks to all of the individuals and groups who purchase and use my goods and services
- similar to 'Dynajail' technology/concept that I was working ('Cloud and Internet Security' Book, p.403) on a while back
- latest in science and technology
- latest in finance and politics
- latest in defense and intelligence
- latest in animal news
- latest in music and entertainment
light show novation launchpad mk2

Random Quotes:
- The Qianwei Fengsheng Paper Company in southwest Sichuan province has teamed up with the China Conservation and Research Center for the Giant Panda to recycle the animal's faeces and food debris into toilet paper, napkins and other household products, state media reported Wednesday.

The goods, soon to be released on the Chinese market, will be marketed as part of a "panda poo" product line decorated with a picture of the bamboo-eating, black-and-white bear.

"They're taking care of our garbage for us," Huang Yan, a researcher at the giant panda centre, told the Chengdu Business Daily.

Huang told Xinhua state news agency that the 10 kilograms (22 pounds) of bamboo poo that adult pandas unleash daily are rich in fibre after absorbing the fructose from the shoots.

In addition to their valuable dung, pandas also produce 50 kilograms of food waste every day from the bamboo husks they spit out after chewing.

While the process of turning bamboo into paper generally involves the breaking down of fructose to extract fibre, this step naturally occurs in the pandas' digestive tract, the paper company's president, Yang Chaolin, told Xinhua.

Fengsheng will collect the faeces from three panda bases in Sichuan a couple of times a week. After it is boiled, pasteurised and turned into paper, it will be tested for bacteria before going on sale.

Boxes of "panda poo" tissues will be sold at 43 yuan ($6.5) a pop.

"Pandas get what they want and we do too," Yang said. "It's a win-win."
- He said three countries among us, namely China , Russian Federation and Iran , rank among the top 6 nations with the largest deposits of natural resources in the world. China and Pakistan alone provide half of the world’s cotton, he boasted.

He said the untapped resources of Afghanistan could make it the richest mining region on earth while its mastery in the infrastructural construction industry has brought Turkey to the top rankings in the world. There is, therefore, all the more reason for us to hold each-others’ hands in support and companionship, he added.
- In an interview with the Washington Examiner on Tuesday, Paul said huge heaps of debt, inflation, and inequality could cause turmoil in the US.

“We’re on the verge of something like what happened in ‘89 when the Soviet system just collapsed,” he said. “I’m just hoping our system comes apart as gracefully as the Soviet system."
- Did a Russian anti-aircraft missile hit one of Israel’s new F-35 stealth fighters?

Pro-Russian media are claiming that an Israeli F-35I was hit and damaged by a Russian-made S-200 surface-to-air missile during an Israeli air strike in Syria earlier this month. Israel says one of its F-35s was damaged—after colliding with a bird.

The story begins on October 16, when Israel announced that its aircraft had struck [3] a Syrian SAM battery near Damascus that had fired two hours earlier on Israeli reconnaissance planes flying over Lebanon. The attack damaged the missile battery, and no Israeli aircraft were hit, according to Israel. Coincidentally or not, the incident happened the same day that Russia’s defense minister, Sergei Shoigu, arrived in Israel for talks with Prime Minister Benjamin Netanyahu and Defense Minister Avigdor Lieberman.
https://scout.com/military/warrior/Article/Did-a-Russian-Missile-Really-Hit-an-Israeli-F-35-112823457
“They said that they were trying to hunt Bin Laden even though the Taliban said in 2001 that they would hand Bin Laden over to the United States if certain protocols were reached and Pakistani authorities would have almost certainly cooperated and so the entire premise for the US presence in Afghanistan was forced to begin with and Bin Laden is of course long gone and according to the assassinated former prime minister of Pakistan Benazir Bhutto, Bin Laden was dead long before the US admitted that he was,” Garrie said.

“What’s really going on is that the United States wants the very rich minerals that lie below the surface of Afghanistan.  It is a very wealthy country below the surface and because of this America has made it a very poor country, and a very dangerous country above the surface,” he added.
- The U.S. has a missile defense system that is a complex web of satellite sensors, radars and interceptors aimed at detecting and destroying incoming warheads. 

"If North Korea were to launch only one missile at us, we could probably shoot it down," said Michael Elleman, a senior fellow for missile defense at the International Institute for Strategic Studies. "But their new missile could carry some very simple decoys, and it’s not certain that the missile we send out will be able to tell the difference between debris, decoys and a real warhead."
- However, Kreil argues that racy candidate numbers simply show how commoditised the CFA exams have become. “The CFA is a mass market exam. It won’t make you any money – it will just show that you’re capable of earning £50-100k ($80k-160k),” says Kreil. “A CFA pass means you’re ticking all the boxes. Employers will see you as a foot soldier whom they can pay a moderate amount and easily replace with someone else who has the same exam passes.”

The CFA Institute doesn’t measure how much people earn upon passing its exams, and was unable to give us any data on this.

Favourably, figures do show that CFA Charter holders are unlikely to be out of work. Equally favourably, in 2012 a CFA compensation survey from the CFA Society of Pittsburgh found the mean salary of society members was $155k and the median was $125k. Less promisingly, our own research earlier this year suggested the average job requiring any kind of CFA qualification pays £66k ($105k). So maybe Kreil was right after all?
- He is an "idiot surrounded by clowns. Trump won't read anything - not one-page memos, not the brief policy papers; nothing. He gets up halfway through meetings with world leaders because he is bored. And his staff is no better."
- He said both companies had waited "until their more clueful customers' complaints reached an adequate volume, and then put up Web pages with mind-numbingly bureaucratic descriptions, that were as difficult to come across by accident as possible.

"This minimally placated the technical community, while avoiding alarming corporate executives, or even making them aware of the problem. With luck, the few who do stumble over the page will not recognise the problem or its severity, from the provided description."

Moen said this approach had been applied in other areas, too. "When Microsoft produced hotfixes that kinda-sorta addressed some (possibly not all) variants of the Ping of Death attack, the patch is classified as 'icmp-fix' and the symptom described as 'A Stop 0x0000000A occurs in Tcpip.sys when receiving Out of Band (OOB) data'.

"Doesn't sound much like 'An anonymous stranger crashes your system by sending it Ping of Death packets from somewhere halfway around the world", does it'?"

He noted in his March 1998 post that one would find less and less useful bug information by delving through FTP sites. "The vendors are tending not to see your serendipitous discoveries as being to their advantage. They prefer to keep a sharp eye on what you find and are interested in, accumulate mandatory survey information on you so they can spam your USPS and e-mailboxes, and present the appropriate spin on whatever knowledge they're obliged to give you.

"Welcome to 1998. We're from the Marketing department. We're here to help you."
- ”I think it is the beginning of a new cold war,” said Mr. Kennan from his Princeton home. ”I think the Russians will gradually react quite adversely and it will affect their policies. I think it is a tragic mistake. There was no reason for this whatsoever. No one was threatening anybody else. This expansion would make the Founding Fathers of this country turn over in their graves. We have signed up to protect a whole series of countries, even though we have neither the resources nor the intention to do so in any serious way. [The NATO expansion] was simply a light-hearted action by a Senate that has no real interest in foreign affairs.”...

...

‘I was particularly bothered by the references to Russia as a country dying to attack Western Europe. Don’t people understand? Our differences in the Cold War were with the Soviet Communist regime. And now we are turning our backs on the very people who mounted the greatest bloodless revolution in history to remove that Soviet regime.

”And Russia’s democracy is as far advanced, if not farther, as any of these countries we’ve just signed up to defend from Russia,” said Mr. Kennan, who joined the State Department in 1926 and was US Ambassador to Moscow in 1952. ”It shows so little understanding of Russian history and Soviet history. Of course there is going to be a bad reaction from Russia, and then [the NATO expanders] will say that we always told you that is how the Russians are — but this is just wrong.” (Emphasis mine.)
- As Matthew Hoh wrote in his 2009 resignation letter from the US State Department in protest of the Afghan War, "The Pashtun insurgency, which is composed of multiple, seemingly infinite local groups, is fed by what is perceived by the Pashtun people as a continued and sustained assault, going back centuries, on Pashtun land, culture, traditions and religion by internal and external enemies… I have observed that the bulk of the insurgency fights not for the white banner of the Taliban, but rather against the presence of foreign soldiers and taxes imposed by an unrepresentative government in Kabul."
- The World Health Organization estimates that between 65 to 80 percent of the world’s population, over 4 billion people, rely on alternative medicine as their primary form of health care compared to only 10 to 30 percent of people who use conventional medicine. 
at
Labels: , , , , , , , ,

Dodgy Job Contract Clauses, Random Stuff, and More

- in this post we'll be going through dodgy job contract clauses. Ironically, many of which are actually unlawful and unenforceable on c...