Preparing Your Code for JDK 9

Preparing Your Code for JDK 9: Erik Costlow discusses what to consider when upgrading to Java 9, where to find early access releases and how to analyze library dependencies for unintentional reliance internal APIs.

Source Code Repositories

In ye olden days of yore, source code repositories were either CVS (Code Verification Store) or SVN (Subversion). Both were horrible at everything: usability, collaboration, stability, merging, committing, branching. Literally everything it was supposed to do. Then along came new technology that did it all properly: Git and Mercurial. Git has officially taken off throughout the world after everyone and their dog has realised how much it works amazingly well. I've used it a lot, and I'm more than happy with it. Git is open source and free, but again like anything, private repos would need to be paid-for hosted or self-hosted. Off-site hosting would be the best, that way you don't have to worry about the cost, time and effort of physical hardware or its administration. 

The most common Git hosted options are: 

GitHub 

GitHub is the world's most popular hosted Git site. It is for a reason: works really well, has a great browser interface and offers an in-built wiki to explain the source code, and issues to track, organizations, user groups, profiles, etc. There is unlimited free allowance for public open-source repositories, so it's great for the scientific community. The main downside is that it's expensive for private use. You're looking at $25/month for a maximum of 10 private repositories (libraries). Eek. 

BitBucket 

BitBucket is the next most popular Git site. Much less so than GitHub, but it's still quite popular. Why? It works just as well as GitHub, but you can create an unlimited number of both public and private repositories. There is a free allowance for up to 5 users on a single private repository. After 5, then you have to pay. On the plus side, if you have an academic email address, if you add it to your BitBucket account (not even as your primary email address) then you automatically get upgraded to the academic features for unlimited everything (not for private Groups though, that should be implemented at the institute-level). This is a perk Atlassian offer in order to encourage, attract and benefit students and faculty. So that's right, BitBucket is made by good ol' Atlassian so it has its own issue tracker, wiki and integrates with the rest of the Atlassian world, i.e. JIRA and Confluence. 

BitBucket can be self-hosted, again with a $10 one-off payment for 10 users. 

CodeCommit 

AWS's Git offering is CodeCommit. Someone else wrote up his review and I couldn't put it better myself: 

Although I’m a fan of CodeCommits simplicity and automation support there are a few negatives to consider before migrating everything to it. There is a cost, albeit a small one, to using it. You’re charged by number of users accessing your repos and the number of git operations. I’m assuming I’m not going to hit the operations limit often but I will immediately be paying for an extra user or two. I’ll have my own AWS CodeCommit user for normal read/write repo access and I’ll need additional, locked down, users for my CI services. Access control is handled using IAM, this is both a positive and a negative depending on how much you like the JSON based policy language. There’s currently no concept of a public CodeCommit repo so you can’t start a code base off in private and then promote it to public view as you can with GitHub or BitBucket. The dashboard UI lacks the familiar collaboration features, there’s no pull requests, issue tracking or wiki built in. If you’re working in a team you’ll need a separate way to handle those. These current limitations means GitHub doesn’t need to be worried just yet, the lack of collaboration features alone will prevent most people from considering CodeCommit as an option even for private only repos. 

From <http://www.unixdaemon.net/cloud/aws-codecommit-initial-thoughts/>  

Basically, it's just not worth it in a world where you have BitBucket and GitHub.

Website solution stacks

In summary, the normal options about how to create a website on a platform are:

1. Traditional LAMP stack, or a variant on it, e.g. Python (Django) instead of PHP, Golang instead of Apache and PHP, PostgreSQL instead of MySQL.
2. Normal MEAN stack, or a variant on it, e.g. DynamoDB instead of MongoDB, Ember.js instead of Angular.js.
3. Serverless, use AWS Lambda to execute Node.js or Java code while running Angular/Ember.js on the client-side.

The LAMP stack

The traditional way of making a website would be through an open-source, free "LAMP" solution stack: https://en.wikipedia.org/wiki/LAMP_(software_bundle)

LAMP stands for:

• Linux - the operating system (OS) of the server.
• Apache - the web server running on the OS.
• MySQL - the relational database the web server/web app connects to for its information
• PHP - the web application / programming language that builds/displays the website, running on the web server.

You can substitute any part for an alternative if desired. E.g.

• Windows instead of Linux (WAMP stack).
• A Python server instead of Apache.
• Oracle or PostgreSQL instead of MySQL.
• Python or Perl instead of PHP.

You can get automatic installers to install the whole stack together, or just install them each separately and connect them up without much effort.

So all this would be running on a server, either physical or virtual.

LAMP is widely adopted throughout the world, a recognized "standard". But this has become less favourable in recent times due to the emergence of the MEAN stack. Problems include things like data transformation, supporting 3 different languages, scaling, and peak-demand performance.

The MEAN stack

Should Node.js be the mandatory choice? No.

MEAN is conceptually very similar to LAMP, but was newly coined in 2013: https://en.wikipedia.org/wiki/MEAN_(software_bundle)

MEAN stands for:

• MongoDB - the NoSQL database.
• Express.js - the web application framework that runs on Node.js.
• Angular.js - the client-side JavaScript framework in the browser.
• Node.js - the server-side Javascript environment (V8 engine).

MEAN is becoming widely adopted now because many people have started using NoSQL databases instead of the typical relational databases. Why? For scalability, performance, space, etc. There are many benefits to going with a NoSQL db. MongoDB is one popular open-source NoSQL db available to freely use. Alternatives are Cassandra, REDIS, and Neo4J (Neo4J being by far the least popular) for particular use-cases. AWS also offers its own NoSQL database: DynamoDB.

Not only that, but people have moved away from using PHP (in LAMP). I have not used PHP myself, but after speaking to colleagues with extensive experience, they all complain about it. Words like "horrible", "awful", "a real pain" get used often. And this is from everyone I have asked about it, who have used it. Instead if you look to what developers are starting to use now, or want to use, it's all about Node.js. Node.js is modular, so you can use whatever components you want, and it handles everything really well on both the server-side where everything gets run, as well as on the client-side through Angular.js for further processing. Even though it is relatively new, it has all the features needed to make it a great system to develop on.

Is Node.js overrated? Maybe. It has asynchronous non-blocking I/O! But so does everything else. Node as a server is cool! But so is everything else. Node has NPN libraries available! But so does everything else have its equivalent. Node has great performance! But Node is only single-threaded, not multi-threaded like everything else.

The Meteor framework

An extension to the MEAN stack is through the Meteor framework. It makes handling of Node.js much easier, as well as the connections with MongoDB and on the client with Angular.js.

https://wiki.dandascalescu.com/essays/meteor_js_vs_the_mean_stack

Apparently it takes a lot of the complexity away because normally each part of the stack needs to be configured individually, which can be a nightmare to get going when you deal with so much different technology. Meteor takes care of this so it's really good for a solo developer to take care of everything in a short space of time. A good review, and link to a good video for its merits can be found: https://wiki.dandascalescu.com/essays/meteor_js_vs_the_mean_stack

This will need some investigation but perhaps using a Meteor framework would be preferable over a manual MEAN stack, there appear to many advantages in our situation, and for future scaling.

The Ember JS framework

Another JavaScript framework is Ember.js which has become very popular is recent times. It's apparently great for creating really snappy web applications for great performance and it's become an industry standard just like Angular.js from the original MEAN stack. This is essentially an alternative to using Angular.js, and there are strong augments in favour of Ember.js, or vice-versa.

The Django framework

Django is a Python framework for the web applications, essentially on the server-side. It would run on an Apache web server, and be connected to a relational database. The main aim is to enable better performance and productivity for creating new websites. Esentially this is would be replacing the "P" in LAMP.

The Ruby on Rails (Rails) framework

Finally, another common framework available is Ruby on Rails, or simply "Rails". It uses the Ruby programming language and it will need a web application like Phusion Passenger and Apache or Nginx, connecting to a database. Rails takes various design patterns to heart in order to reduce load on the developer, to make their life easier for dealing with complexity. Again, this would replacing the "P" in LAMP.

The Golang language and related framework

It's possible to use a web application stack using Golang (Go, from Google) as the programming language, optionally also using a Go web framework like Gorilla or GoCraft and a (Go) web application to run the server-side code on. Even then, a framework isn't super needed because Go has lots of good important standard libraries available out of the box without a need for an extra web framework.  Go can also work alongside other frameworks, like Angular.js. Go has libraries to connect to all AWS services too. The downside to choosing Go would be that it's an emerging/niche language since it's not so popular compared to Node or JS in general. So getting someone in might be a bit more difficult further down the line, but in reality, if you could find someone who has either experience or willing to learn/tackle Go, then you're almost certainly set with a very good programmer. Go has amazing performance (much better than Node.js), and you can even run your web server with a Raspberry Pi.

Go should be a serious contender, couldn't put it any better than this comment:

… if you want to do something like writing a tool, a server, a web application, a network application or an application to process, send/receive something or whatever that is called high-level. Things that previously people have to use such slow and unsafe scripting languages like Python, Ruby or PHP to write. You don't care about GC, you don't care about memory management, lifetimes and all that stuff, and you shouldn't care because Go is a compiled language which is more than fast enough to do ten times the work. You just want something simple, fast, compiles instantly and makes your code the easiest to read, to understand, to cooperate and to maintain. Choose Go, for the love of god choose Go because it's awesome for the job.

From <https://www.reddit.com/r/golang/comments/29ua55/rust_or_go/ciojqcg>

Big companies like Twitter are also using Go now, but big companies pretty much use a every technology somewhere.

Server-less architecture

The problem with both the LAMP and MEAN/Meteor stacks are that it requires you still to need to host/run/administer/backup/fix a virtual machine, web server, and database all by yourself. This is all in addition to the actual programming that needs to be done to write the web application. Another architecture type is available that eliminated the need for the virtual machine and web server: serverless.

Another, less popular way to host a website is directly through cloud services like AWS S3 cloud using Lambda functions . This eliminates the need for a web server, like how it could for (1) above. Here, static content would be hosted on S3, as well as dynamic JS. JS then gets executed on the client-side (browser), which then asks for further information through to Lambda, which will then query to the database or S3 storage by executing code (JavaScript or Java). Lambda would essentially replace the need for an web server running 24/7, and AWS would bill you according to how often Lambda was used (100ms execution timeframes).

https://d0.awsstatic.com/whitepapers/AWS_Serverless_Multi-Tier_Architectures.pdf

For this type of implementation, the price should be a factor as well -t he cost of Lambda functions vs the cost of running, testing, administering an web server for the web app.

Not many people have adopted this style because people either aren't convinced it can scale (it can), or they would rather go for an industry recognised design/stack/architecture, or for cost reasons. I'm sure nobody got fired from a big company or a start-up for going with a LAMP stack, or a MEAN stack with Angular, or for using Ember. Telling your boss you want to go with a new serverless design that almost nobody else does in order to decrease costs and increase performance plus scalability? That might not go over too well, purely because it's seen to be riskier and unproven since not many people decided to do it.

…it’s a complete package for building a real, scalable backend service and fronting it with both mobile clients and a website, all without the need for servers or other infrastructure in any part of the system: frontend, backend, API, deployment, or testing. Go serverless!

From <https://aws.amazon.com/blogs/compute/microservices-without-the-servers/>

Serverless Microservices: From <https://www.youtube.com/watch?v=w14NJkV5yAg>

Example implementations:

• http://www.infoq.com/articles/mars-rover-application-DynamoDB
• http://highscalability.com/blog/2015/12/7/the-serverless-start-up-down-with-servers.html

Side Notes

For the MEAN stack, the MongoDB would have to be hosted somehow, just like the web application has to be. MongoDB has a free open-source licence, but like anything apart from AWS Lambda, it would need to be hosted somewhere. MongoDB needs multiple nodes (instances) to operate effectively. Having to manage and backup this manually might be a real pain and cause a lot of extra effort for not much gain, e.g. primary/secondary setup, problems if nodes go down, problems scaling and adding more nodes, etc. A hosted solution might be less of a headache than self-hosting.

What are AWS Cloud Design Patterns?

The AWS Cloud Design Patterns (CDP) are a collection of solutions and design ideas for using AWS cloud technology to solve common systems design problems. To create the CDPs, we reviewed many designs created by various cloud architects, categorized them by the type of problem they addressed, and then created generic design patterns based on those specific solutions. Some of these problems could also be addressed using traditional data-center technology, but we have included cloud solutions for these problems because of the lower cost and greater flexibility of a cloud-based solution.

http://en.clouddesignpattern.org/index.php/Main_Page

Robert C Martin - Functional Programming; What? Why? When?

Go programming language reading list

I found a really good reading list of books for the Go programming language: https://github.com/dariubs/GoBooks
Apparently the documentation for Go is very good on the website itself, which of course is all freely available and well laid out: https://golang.org/
I aim to check this out in more detail when I have the time, which might be a while away since I've got other priorities right now.

Finished reading: The Passionate Programmer

I finally finished reading (and making notes) about The Passionate Programmer. It was an excellent read and I would recommend it to anyone who needs advice on how to improve their career in software development / engineering. 10/10

I've got a big action plan about how I should be improving myself both in and outside of work. I've had thoughts about this before in general, but the book really spells it out  about the techniques on how to make small and large career improvements in terms of technical development, direction, etc. I really wish I had read this book after I had graduated my undergraduate degree, I feel like I've been left behind for the last decade.

That took me 2 weeks of sustained conscious effort to get through the book, which is only 200 pages long. And so I have a thousand things to do and explore on how to build myself and my career up. It's exciting and fun, I'm a bit concerned that I don't know how I'm going to devote time during a typical week (and exercise! And hobbies!) into the week. Well, guess I'll find out sooner or later how I'm going to manage my time.

Agile development

I found a good site to read more about Agile development, and the various parts and processes involved:
https://www.atlassian.com/agile

I'd like to improve on the way we approach agile development within my team at work, so hopefully this site will explain the concepts well and give some insight.

Google Go introductory video

I wanted to know what Google's Go programming language was about, and I found this really good introductory video about it:
http://www.infoq.com/presentations/Go-Google
It's a presentation ans slides format, and I love how the screen layout has the slides automatically next to the presentation video right there - very useful

Also this FAQ is very comprehensive:
https://golang.org/doc/faq

Reading programming books

I've always struggled reading technical books, and I'm looking for ways to improve myself upon this. With my recent reading list I've got a opportunities to address this and better myself.

In the past what might happen is: I buy a textbook that seems to have good reviews about what I want to know more about, then I try and read it and can get barely through a chapter. Then I try other chapters that I may be more interested in and the same thing happens again. Then the fat book ends up on the bookshelf gathering dust. Time passes and the technology has moved on and it's obsolete, or I'm never going to use that technology and lost all drive about it.

The reasons for that are because the information is technically difficult to understand, and to know about it properly, I feel it needs to be applied, in say tutorial examples, or real-life work. And more often then not I either can't get the tutorials to work, the tutorials are too basic, and there are no work opportunities for its application. Knowledge or skills are barely obtained in the first place, and are then not retained at all, and frustration kicks in.

I know there are generally two types of programming books: guides and references. So when I'm only referring to guides, and that's what I'm really interested in learning new technologies, and not looking up particulars.

I've known people who can literally read a technical book from start to finish and immediately assimilate the knowledge. They remember the content and then are capable of immediately applying to to a situation, perhaps referring to the material or perhaps not sometimes. I have tried to do that, and I am simply incapable of doing so, which again frustrates me.

I remember when I was trying to get into Spring Batch more, and I would aim to read and complete chapters every week for a couple months. Then in work, using Spring Batch, I could apply almost nothing from my "homework" because I wouldn't remember anything that I had read about.

My strategy was reading and making notes (and re-reading, re-reading) and completing tutorials, but I think in my case I need more than that, and I'm not sure what. I have tried to identify better strategies and I found a few posts online about that:

• https://blog.newrelic.com/2015/03/18/learn-code-programming-book/
• http://programmers.stackexchange.com/questions/17826/how-do-people-read-big-technical-books
• http://norvig.com/21-days.html

I think a lot of this comes down from my school or university days. I mean, in my lifetime I've received no formal training in how I am supposed to learn from a textbook, especially programming. So perhaps I'm doing things wrong. Perhaps smarter people than me don't need to follow other learning protocols, or the best practice for learning have already come naturally to them. Muddling along and trying to do what I've done before isn't for the best, and I'm tired of it.

Software deveopment reading list

Starting my new year's resolution a month late, I've put together a reading list I want to start getting through this year. It focuses on software development / engineering, and also a little about Java too.

Helping conduct recent interviews has prompted me to make a start on this, because refreshing my mind about concepts, design patterns, etc would be a good idea. It certainly can't hurt, and it would definitely help as I make the transition to a software developer proper in my workplace. Apart from technical knowledge, I'm sure the material also explains a lot about how to approach work better, or be more productive through work.

The list contains some Java-related books too, this is because most of our code at work is using Java 7, and I haven't had the need to, or exposure to, 8 yet. This way hopefully I can get more up to speed with recent developments I've not had to chance to consider in previous months. Java 9 is already progressing, so it makes a lot of sense to drive on with this too.

I thought I'd share my reading list, in case someone else would be in a similar situation and looking for a place to start. By no means is this a complete list, and I'd be open to any suggestions as well for any other good material.

• The Pragmatic Programmer
• The Passionate Programmer
• Clean Code
• The Clean Coder
• The well-rounded Java developer
• Design patterns: elements of reusable object-oriented software
• Head First, Design Patterns
• Introduction to Algorithms
• Data Structures and Algorithms
• Java SE 8 for the really impatient
• Java 8 in Action
• Spring Data

What started me on making my list was this very informative post:

http://blog.codinghorror.com/programmers-dont-read-books-but-you-should/