Broadcasts: Alert the community when you're available for work

Starting today, developers can broadcast their work availability to the community. Hiring managers receive matches based on the candidate's skills and location.

What this means for developers: When you're looking for a new contract or full-time opportunity, share it to the Moonlight community to start getting interviews!

What this means for hiring managers: Solve your sourcing problems by getting a feed of active developer matches! Make sure the skills on your company profile are up-to-date so you can get the most relevant matches.

updates.png

Slack registration is fixed

Most users of Moonlight choose to use our Slack integration rather than in-app chat to connect with others. It's a convenient way to participate in the community using your existing workflows!

Unfortunately, we missed an API change in Slack that broke how our registration process works. So, Slack signups were not being connected to Moonlight accounts.

Today we fixed that problem!

If you joined the Slack community but were not added to any private channels (e.g., #companies or #developers) - then please email team@moonlightwork.com and we will manually connect your account!

(If you're not in the Slack community - click "Chat" in the app and find a sign-up link in the corner!)

Better search results for skills

We learned a lot about best practices for autocomplete in this excellent article. So, we just released an update to our skill search component that applies many of these best practices!

The skill search component is used in many places throughout the app. Moonlight users see it when they post a new job, add skills to their profile, or search for a developer. We most recently updated it by incorporating some machine learning.

React is one of the most popular skills on Moonlight. Let's see the improvements in action by typing "Re" into the search component:

Old results (Bad)

Old autocomplete algorithm

Updated results (Better)

Screen Shot 2019-03-22 at 12.04.03 PM.png

Summary

As you can see, it prioritizes matches that start with rather than just contain the query make a huge improvement. Also, we show "React" before "React Native" because it is shorter.

We added a couple of tweaks beyond those described in the article. For example, our algorithm detects searches that are acronyms of skills (so that "AWS" returns "Amazon Web Services" first!)

If you notice any weird behavior, please let us know!

Launching a weekly newsletter

Tomorrow we are launching a weekly newsletter for clients and developers on Moonlight. We hope it can decrease email noise, and better let you know what is most relevant and helpful to you on Moonlight!

Benefits for everybody:

  • A digest of your Moonlight action items
  • See latest community posts and updates

Benefits for developers:

  • Review latest job matches
  • Add latest skills to your profile for better matching

Benefits for companies:

  • Review new and recommended developer matches
  • Get status updates about open job posts

This is an experimental system that we built in-house. Let us know your thoughts and feedback, plus any bugs you find!

One-click unsubscribe

It's now easier to manage the emails that you receive from Moonlight. Marketing emails now include a one-click unsubscribe link in their footer.

Manage your email subscriptions, and see the type of emails that Moonlight sends on the settings page.

Cleaner email templates

We've updated our standard email template! It looks cleaner and more consistent with our latest branding.

If the layout happens to render incorrectly in your email client, please let us know so that we can fix it.

Old:

Screen Shot 2019-02-26 at 3.25.57 PM.png

New:

Screen Shot 2019-02-26 at 3.26.34 PM.png

Referrals: Invite and earn $500

Moonlight's professional community is made up of thousands of developers, designers, product managers, and founders. We value our existing members and want to fairly incentivize you for growing the network. So, today we are updating and expanding our referral program!

💼 Clients can now participate in the referral program

Clients of Moonlight can now set up a referral link and get paid for referring new users. Just set up payments to activate your link.

💰 Increasing the client referral payout to $500

Get $500 when a new client signs up through your referral link and becomes a paying member. We'll send cash to your connected Stripe account automatically!

👷‍♀️New reward program for developer referrals

Get Moonlight swag for every developer you refer! We'll send you a gift from the remote work store when new developers sign up through your link and are accepted to the community.

➡️ Get started

Get your personal links to share on the referrals page of the Moonlight website!

Get your referral link

Moonlight Referral Program

Q&A

What does a client have to spend to trigger a payout?

They need to spend $1000 on memberships or job posts. With our current pricing, this means that any single purchase will trigger a payout.

Can I send an email introduction?

Yes! Just send an introduction email to founders@moonlightwork.com, and we will credit you for resulting signups. Copy our recommended intro message on the referrals page!

Outage post-mortem

From January 18 to January 22, Moonlight's website and API had intermittent outages. These started as occasional API errors but culminated in periods of downtime yesterday. The problems have been resolved, and the application has returned to normal.

Background

Kubernetes is the application hosting software that Moonlight uses. Kubernetes runs applications on groups of servers. The servers are called nodes. Copies of an application that run on the node are called pods. Kubernetes has a scheduler that dynamically decides which pods should run on which nodes.

Timeline

The original errors on Friday appeared to be connectivity problems with a Redis database. Moonlight's API uses Redis on every authenticated request to validate sessions. We noticed that our Kubernetes monitoring reported that some nodes and pods were unresponsive. Our hosting provider Google Cloud reported networking service disruptions, and we assumed that the outage was the root cause.

As traffic declined over the weekend, the errors seemed to resolve with only occasional problems. On Tuesday morning, the Moonlight website was offline with zero external traffic reaching the cluster. Finding another person on Twitter with similar symptoms led us to believe that Google's hosting was experiencing a networking outage. We contacted Google Cloud Support, who quickly escalated the outage to their support engineering team.

Google's support team identified a pattern in the nodes of our managed Kubernetes cluster. When the nodes experienced periods of sustained 100% CPU usage, the virtual machine would experience a kernel panic and crash.

Cause

The cycle that caused the outage appeared to be:

  1. Kubernetes scheduler assigned multiple pods with high CPU use to the same node
  2. The pods consumed 100% of the CPU resources on the shared node
  3. The node experienced a kernel panic, resulting in a period of downtime where the node appeared unresponsive to the scheduler
  4. The scheduler would reassign all of the crashed pods to a new node, which repeated the same process - and compounded the effects

Initially, only the Redis pod was experiencing this crashing error. But, eventually, all pods serving web traffic were going offline, resulting in a complete outage. Exponential back-off rules resulted in longer periods of downtime.

Fix

We were able to restore website functionality by adding anti-affinity rules to all of our major deployments. This spreads the pods out across nodes automatically, which increases failure tolerance and improves performance.

Kubernetes is designed to be a fault-tolerant hosting system. Moonlight runs three different server nodes for fault tolerance, and we run three copies of every web-facing application. The intention was to have one copy on each node, which would allow two node failures before downtime. However, Kubernetes occasionally scheduled all three website pods on the same node, which created a single point of failure. Other CPU-intense applications (specifically - server-side rendering) were being scheduled onto the same node, rather than separate ones.

A properly-functioning Kubernetes cluster should be able to manage sustained periods of high CPU usage and move pods around to utilize available resources efficiently. We are continuing to work with Google Cloud support to identify and fix the root cause of the server kernel panics.

Conclusion

Anti-affinity rules make web-facing Kubernetes applications more fault-tolerant. If you run a user-facing service on Kubernetes, you should consider adding them.

We are continuing to work with Google to identify and fix the cause of the node kernel panics.

Clients can now buy single job posts

We recently launched membership options for clients. We have added the ability to purchase a single job post without a subscription.

Single job posts do not give access to Moonlight's full functionality, such as search, cold outreach to developers, and direct hiring. But, they are a great way to test Moonlight before committing to a membership.

New homepage

We redesigned the Moonlight homepage! Check it out and let us know if you see any problems.

moonlight-image.png

No published changelogs yet.

Surely Moonlight will start publishing changelogs very soon.

Check out our other public changelogs: Buffer, Mention, Respond by Buffer, JSFiddle, Olark, Droplr, Piwik Pro, Prott, Ustream, ViralSweep, StartupThreads, Userlike, Unixstickers, Survicate, Envoy, Gmelius, CodeTree