Spent several hours trying to upgrade my Ubuntu installation from 15 up to the latest 17. The upgrade didn’t fail, but I did see a few error messages, and now I have applications failing to start for various reasons, including the settings applet; and when I install or use my nvidia drivers, ubuntu doesn’t start up properly until I do
[code]
apt-get purge nvidia*
[/code]
But removing all the nvidia stuff causes it to fallback to nouveau which for the most part works, but not exactly good for any linux gaming.
Looks like it’s going to be a full-reinstall job to make sure everything is clean :(
Caveat though – you will have to make sure that your node security group allows your bastion security group to talk to the nodes on the additional ports. By default, the only port that the bastions are able to talk to the node security groups on is SSH (22) only.
Let’s assume you have an application that runs happily on its own and is stateless. No problem. You deploy it onto Kubernetes and it works fine. You kill the pod and it respins, happily continuing where it left off.
Let’s add three replicas to the group. That also is fine, since its stateless.
Let’s now change that so that the application is now stateful and requires storage of where it is in between runs. So you pre-provision a disk using EBS and hook that up into the pods, and convert the deployment to a stateful set. Great, it still works fine. All three will pick up where they left off.
Now, what if we wanted to share the same state between the replicas?
For example, what if these three replicas were frontend boxes to a website? Having three different disks is a bad idea unless you can guarantee they will all have the same content. Even if you can, there’s guaranteed to be a case where one or more of the boxes will be either behind or ahead of the other boxes, and consequently have a case where one or more of the boxes will serve the wrong version of content.
There are several options for shared storage, NFS is the most logical but requires you to pre-provision a disk that will be used and also to either have an NFS server outside the cluster or create an NFS pod within the cluster. Also, you will likely over-provision your disk here (100GB when you only need 20GB for example)
Another alternative is EFS, which is Amazon’s NFS storage, where you mount an NFS and only pay for the amount of storage you use. However, even when creating a filesystem in a public subnet, you get a private IP which is useless if you are not DirectConnected into the VPC.
Another option is S3, but how do you use that short of using “s3 sync” repeatedly?
One answer is through the use of s3fs and sshfs
We use s3fs to mount the bucket into a pod (or pods), then we can use those mounts via sshfs as an NFS-like configuration.
The downside to this setup is the fact it will be slower than locally mounted disks.
This article is a guide on how to setup a Kubernetes cluster in AWS using kops and plugging it into your own subnets and VPC. We attempt to minimise the external IPs used in this method.
Export your AWS API keys into environment variables
Create the cluster (you can change some of these switches to match your requirements. I would suggest only using one worker node and one master node to begin with and then increase them once you have confirmed the config is good. The more workers and master nodes you have, the longer it will take to run a rolling-update.
Important note: There must be an ODD number of master zones. If you tell kops to use an even number zones for master, it will complain.
If you want to use additional security groups, don’t add them yet – add them after you have confirmed the cluster is working.
Internal IPs: You must have a VPN connection into your VPC or you will not be able to ssh into the instances. The alternative is to use the bastion functionality using the –bastion flag with the create command. Then doing this:
However, if you do this method, you MUST then use public IP addressing on the api load balancer, as you will not be able to do kops validate otherwise.
If you have a VPN connection into the VPC, change spec.api.loadBalancer.type to “Internal”, otherwise, leave it as “Public”
Change spec.subnets to match your private subnets. To use existing private subnets, they should also include the id of the subnet and match the CIDR range, e.g.:
The utility subnet is where the Bastion hosts will be placed, and these should be in a public subnet, since they will be the inbound route into the cluster from the internet.
If you need to change or add specific IAM permissions, add them under spec.additionalPolicies like this to add additional policies to the node IAM policy (apologies about the formatting. WordPress is doing something weird to it.)
Edit the bastion, nodes, and master configs (MASTER_REGION is the zone where you placed the master. If you are running a multi-region master config, you’ll have to do this for each region)
If you want a mixture of instance types (e.g. t2.mediums and r3.larges), you’ll need to separate these using new instance groups ($SUBNETS is the subnets where you want the nodes to appear – for example, you can provide a list “eu-west-2a,eu-west-2b)
(You can apply the yaml directly onto the cluster via the dashboard. Make sure you change the filter to match your domain or subdomain. )
Note that if you use this, you’ll need to change the node IAM policy on the cluster config as the default IAM policy won’t allow the External DNS container to modify Route 53 entries, and also annotate (use kubectl annotate $service_name key:value) your service with text such as:
And also you might need this annotation, to make the ELB internal rather than public - otherwise Kubernetes will complain “Error creating load balancer (will retry): Failed to ensure load balancer for service namespace/service: could not find any suitable subnets for creating the ELB”
It will allow you to visually see a topology of your cluster at a cluster and also provides some management features too. For example, here’s my cluster. It contains 5 nodes (1 master, 4 workers and is running 4 services (Kubernetes, external-dns, cockpit, and dashboard). Cockpit creates a replication controller so it knows about the changes.
Add any additional security groups by adding this under the spec section of the node/master/bastions config, then do a rolling-update (you might need to use the –force switch), do this as soon as you can after creating and verifying the cluster updates work.
[code lang=text]
additionalSecurityGroups:
sg-xxxxxxxx
sg-xxxxxxxx
[/code]
If the cluster breaks after this (i.e. the nodes haven’t shown up on the master), reboot the server (don’t terminate, use the reboot option from the AWS console), and see if that helps. If it still doesn’t show up, there’s something wrong with the security groups attached – i.e. they’re conflicting somehow with the Kubernetes access. Remove those groups and then do another rolling-update but use both the –force and –cloudonly switches to force a “dirty” respin.
If the cluster comes up good, then you can change the node counts on the configs and apply the update.
Note that if you change the node count and then apply the update, the cluster attempts to make the update without rolling-update. For example, if you change the node count from 1 to 3, the cluster attempts to bring up the 2 additional nodes.
Other things you can look at:
Kompose - which converts a docker-compose configuration into Kubernetes resources
dig the nameservers for the hosted zone you created
dig NS k8stest.blenderfox.uk
If your internet connection already has DNS setup to the hosted zone, you’ll see the nameservers in the output:
;; QUESTION SECTION:
;k8stest.blenderfox.uk. IN NS
;; ANSWER SECTION:
k8stest.blenderfox.uk. 172800 IN NS ns-1353.awsdns-41.org.
k8stest.blenderfox.uk. 172800 IN NS ns-1816.awsdns-35.co.uk.
k8stest.blenderfox.uk. 172800 IN NS ns-404.awsdns-50.com.
k8stest.blenderfox.uk. 172800 IN NS ns-644.awsdns-16.net.
Export your AWS credentials as environment variables (I’ve found Kubernetes doesn’t reliably pick up the credentials from the aws cli especially if you have multiple profiles
You can also add it to a bash script and source it.
Create the cluster using kops. Note that the master zones must have an odd count (1, 3, etc.) since eu-west-2 only has two zones (a and b), I have to have only one zone here
While the cluster starts up, all the new records will be set up with placeholder IPs.
NOTE: Kubernetes needs an externally resolvable DNS name. Basically, you need to be able to create a hosted zone on a domain you control. You can’t use Kops on a domain you can’t control, even if you hack the resolver config.
The cluster can take a while to come up. Use
kops validate cluster –state s3://k8sstate
To check the cluster state.
When ready, you’ll see something like this:
Using cluster from kubectl context: cluster.k8stest.blenderfox.co.uk
Validating cluster cluster.k8stest.blenderfox.co.uk
INSTANCE GROUPS
NAME ROLE MACHINETYPE MIN MAX SUBNETS
master-eu-west-2a Master m4.large 1 1 eu-west-2a
nodes Node m4.large 3 3 eu-west-2a,eu-west-2b
NODE STATUS
NAME ROLE READY
ip-172-20-35-51.eu-west-2.compute.internal master True
ip-172-20-49-10.eu-west-2.compute.internal node True
ip-172-20-72-100.eu-west-2.compute.internal node True
ip-172-20-91-236.eu-west-2.compute.internal node True
Your cluster cluster.k8stest.blenderfox.co.uk is ready
Now you can start interacting with the cluster. First thing is to deploy the Kubernetes dashboard
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard.yaml
serviceaccount "kubernetes-dashboard" created
role "kubernetes-dashboard-minimal" created
rolebinding "kubernetes-dashboard-minimal" created
deployment "kubernetes-dashboard" created
service "kubernetes-dashboard" created
Now setup a proxy to the api
$ kubectl proxy
Starting to serve on 127.0.0.1:8001
Next, access
http://localhost:8001/ui
To get the dashboard
Now let’s create a job to deploy on to the cluster.
The classic rm -rf / is there, along with accidental dd‘ing or mkfs‘ing the wrong disk (I’ve done that before), but the lesser known fork bombs and moving to /dev/null are in there (I often redirect output to /dev/null, but not moved files into there. That’s an interesting way of getting rid of files.
Not often I quote from a publication from Ireland, but this was quite an intriguing read. Someone who went from Windows to Mac to Linux (Mint)
Linux is everywhere – and will free your computer from corporate clutches
It was 2002, I was up against a deadline and a bullying software bubble popped up in Windows every few minutes. Unless I paid to upgrade my virus scanner – now! – terrible things would happen.
We’ve all had that right?
In a moment of clarity I realised that the virus scanner – and its developer’s aggressive business model – was more of a pest than any virus I’d encountered. Microsoft’s operating system was full of this kind of nonsense, so, ignoring snorts of derision from tech friends, I switched to the Apple universe.
It was a great choice: a system that just worked, designed by a team that clearly put a lot of thought into stability and usability. Eventually the iPhone came along, and I was sucked in farther, marvelling at the simple elegance of life on Planet Apple and giving little thought to the consequences.
Then the dream developed cracks. My MacBook is 10 years old and technically fine, particularly since I replaced my knackered old hard drive with a fast new solid-state drive. So why the hourly demands to update my Apple operating system, an insistence that reminded of the Windows virus scanner of old?
Apple is no different to Microsoft it seems.
I don’t want to upgrade. My machine isn’t up to it, and I’m just fine as I am. But, like Microsoft, Apple has ways of making you upgrade. Why? Because, as a listed company, it has quarterly sales targets to meet. And users of older MacBooks like me are fair game.
I looked at the price of a replacement MacBook but laughed at the idea of a midrange laptop giving me small change from €1,200. Two years after I de-Googled my life(iti.ms/2ASlrdY) I began my Apple prison break.
He eventually went for Linux Mint, which for a casual user is fine. I use Fedora and Ubuntu (and a really old version of Ubuntu since my workplace VPN doesn’t seem to work properly with anything above Ubuntu 14 - their way of forcing me onto either a Windows or Mac machine)
As someone who has tinkered with multiple distributions, this will be a great way to try out multiples
This tutorial shows you how to install multiple Linux distributions on one USB. This way, you can enjoy more than one live Linux distros on a single USB key.
Torvalds is not a huge fan of the ‘security community’ as he doesn’t see it as black and white. He maintains that bugs are part of the software development process and they cannot be avoided, no matter how hard you try. “constant absolute security does not exist, even if we do a perfect job,” said Torvalds in a conversation with Jim Zemlin, the executive director of the Linux Foundation.
“As a technical person, I’m always very impressed by some of the people who are attacking our code,” Torvalds said. “I get the feeling that these smart people are doing really bad things that I wish they were on our side because they are so smart and they could help us.”
Looking forward to when LineageOS can upgrade to Oreo. There’s a lot of new features that may make life a lot easier generally. Take a look in the article for details
We take a 20,000 word deep-dive on Android's "foundational" upgrades.
However, also notable and I’m really happy about this:
Regarding installation, we recommend that users wipe when switching to LineageOS, and reinstall their gapps. However, we recognize that this can be time consuming, so we are offering an EXPERIMENTAL (read as, if it fails, you’ll have to wipe anyways) solution.
Alongside the ‘weekly’ release for your supported device, we’ll provide an EXPERIMENTAL data migration build.
This build will allow you to ‘upgrade’ from CM to the signed LineageOS weekly
This build may wipe permissions (you’ll have to re-allow app permissions), but should retain all user data
This build will be watermarked with an ugly banner to ensure that you don’t permanently run this EXPERIMENTAL release, and upgrade to a normal weekly after.
The process for this installation will be as follows:
Install EXPERIMENTAL migration build on top of cm-13.0 or cm-14.1 build (don’t try to install LineageOS 13.0 on top of CM 14.1, that will not work).
Reboot
Install LineageOS weekly build
Reboot
Re-setup your application permissions
Given the EXPERIMENTAL nature of this process, we are going to remove this option in two months time.
Despite being a library that most people outside of the technology industry have never heard of, the Heartbleed bug in OpenSSL caught the attention of the mainstream press when it was uncovered in April 2014 because so many websites were vulnerable to theft of sensitive server and user data. At LinuxCon Europe, Rich Salz and Tim Hudson from the OpenSSL team did a deep dive into what happened with Heartbleed and the steps the OpenSSL team are taking to improve the project.
An attempt to bust some of the myths that surround Linux. Not a lot of them, but still some of them - some of which I see a lot in Windows communities. And the old classic “Linux is CLI only” (facepalm)
Linus has his moments. He’s well-known for having a short-temper, lashing out at a contributor, but he is also known for creating Linux which a significant number of devices these days use, in some form or another. I work with it on a daily basis both at work and at home. Without this guy, I wouldn’t be where I am now. Well, possibly I would still be here, but dealing with cough Windows cough servers instead….
On August 25, Linux creator Linus Torvalds will be in a plane somewhere between Canada and the United States as his handiwork, which has completely changed the world of computing, marks its 25th birthday.
Steam was one of the many things that broke with Ubuntu 16.04 because of numerous changes in package names and dependencies. Fortunately, here’s a guide to fix that. Now, back to my Dungeon Defenders :D
Everyone loves hearing about pentesting and ethical hacking distros these days, and it looks like it is even becoming a trend among aspiring security professionals.
Therefore, today we have some good news for those who want to try one of the best penetration testing and security auditing operating systems based on the Linux kernel, Kali Linux, the successor of the popular BackTrack, and don’t have the resources to run the Live CD or install the OS on their computers.
Network security specialist Jerry Gamblin has created a project called KaliBrowser, which, if you haven’t already guessed, it allows you to run the famous Kali Linux operating system on a web browser, using the Kali Linux Docker image, Openbox window manager, and NoVNC HTML5-based VNC client.
Looks like my weekend is going to be filled with tinkering again. ^_^;
I need to reinstall windows on my laptop as I think there must be some graphics conflict somewhere and it’s lagging when it gets taxed (didn’t normally). Most commonly, it happens when I’m playing Final Fantasy XIV, but has lagged a bit on Alice: Madness Returns and Hyperdimension Neptunia U: Action Unleashed. I figured it might be my connection, since FFXIV is an MMORPG, so I switched from my WiFi to my 4G connection via tethering and it still lags. I then switched from DirectX 9 to DirectX 11, amd still nothing. I even downgraded my Nvidia driver to a REALLY old version (since Nvidia ran into a huge bug with one of their drivers, if you recall), so I’m planning to run my Clonezilla backup tonight (which should take a few hours since I’m also backing up my Ubuntu install), and then run my Windows install then then boot-repair to get grub back (凸(>皿<)凸 Microsoft)
And then, I have to go through the process of installing drivers and updating Windows, though I will probably skip updating Windows since I only use it as a gaming environment. And downloading my Steam games again. Including the Heavensward expansion, Final Fantasy XIV is probably about 20-30GB. With the spikes and dips in download speed on my 4G, it’s going to take about 3 hours.
Whilst I totally respect Mark for coming out and saying this, that’s not to say that in future, Canonical could be bullied into implementing a back door, or Ubuntu cracked by some untoward government agency….
VIDEO: Mark Shuttleworth, founder of Canonical and Ubuntu, discusses what might be coming in Ubuntu 16.10 later this year and why security is something he will never compromise.
Jack Wallen reviews the bq Aquaris M10 tablet and he’s impressed. If you’ve been on the fence about Ubuntu Touch, this might just assuage those unpleasant feelings.
Canonical tried to do this with their last attempt to crowdsource their Ubuntu phone, but it didn’t make enough money. This one looks pretty good too. Now I wonder if I could run Android apps on there too. :D
I decided to make a second part of my Linux gaming setup post, as I feel that the first one is more like a list of stuff that are on top my desk. Anyway, once you’re done with hardware, let&#…