New tutorial! How to register custom platforms in Sugar instances

If you have a REST API integration with Sugar, you are likely using a custom platform as part of that integration.  Beginning in the Winter ‘18 release, Sugar is enforcing a rule that all custom platforms must be registered in each Sugar instance where a custom platform is used.  For more details on the rule itself, see Matt’s earlier blog post: Unknown platforms to be restricted in Winter ‘18 release.

The bad news?  This change that requires all custom platforms to be registered probably affects you.

The good news?  There is an easy fix, and I’ve written a tutorial that walks you step-by-step through how to implement the fix.

Check out the tutorial here!

How Sugar REST API versioning works

Sugar v11 REST API

In order to support some cool new dashboard features, we made some enhancements to our Dashboard APIs in the Fall ’17 release. So we added a new REST API version, v11, in that release in order to ensure clients and integrations that were using the existing v10 Dashboard APIs would continue to work properly. For more details on the v11 API, we recommend you check out a recording of our recent Fall ’17 release developer webinar and the related migration guide.

Today we won’t focus on version 11 specifically but instead on how the Sugar platform will be handling REST API versioning in the future. So read on!

Passing API version using HTTP Header

For REST v10 and our older web services frameworks, the API version was specified exclusively via the URL.

Example 1)

GET <sugar>/rest/v10/me

In the Fall ’17 (7.10) release, we introduced an additional way to specify the API version: using an HTTP header. This change allows resource URLs to be treated as unique identifiers to resources regardless of the API version in use.

The REST API version can now be specified via HTTP Accept header. The REST API version should be specified in either the URL or in the HTTP Accept header but not in both locations at once. This new method of specifying the API version in the HTTP Accept header better adheres to RESTful architectural principles. By essentially encapsulating the API version as part of media type, a single resource at a given URL could offer multiple representations using different data formats and API versions.

Example 2)

GET <sugar>/rest/me
...
Accept: application/vnd.sugarcrm.core+json; version=42

We recommend that client start specifying the API version in the Accept header instead of as a component of the URL. Sugar clients will adopt use of the Accept header in the future.

Future use of semantic versioning in REST API

As you probably have noticed, the REST API version is a property of the Sugar REST API as a whole and not individual endpoints.

We plan to implement a semantic versioning policy with MAJOR.MINOR format for the REST API next year.

The version will be specified in the request Accept header as MAJOR.MINOR (ex. 11.1) whereas the version in the endpoint url will be specified as MAJOR_MINOR (ex. 11_1).

The Sugar REST API MAJOR version will be incremented for each Sugar release where compatibility is being broken on any endpoint. The Sugar REST API MINOR version will be incremented for each Sugar release where an API endpoint is changed or additional API endpoints are being added without breaking compatibility.

Not every new Sugar release will result in new REST API versions. Our intent is to only add new features or make breaking changes to our REST API only when it is not possible or prudent (ex. for performance reasons) to do so with the current API.

When there is a breaking change, older API versions should continue to work as-is. For example, if a parameter was removed in newer version, the old version should still support that parameter.

What are breaking and non-breaking changes?

We want to be clear about what we consider to be an API breaking change and what is not.

Breaking Changes

• Removing or renaming an endpoint, a request parameter, or a fixed response field
  • For example, removing the GET /rest/v10/<module> endpoint or renaming the _acl field to _access_control.
• Removing support for a HTTP request method on an endpoint
  • For example, changing from PUT to PATCH method for updating records.
• Change in type for a request parameter
  • For example, a change from an Integer to Decimal value.
• Change in response field format
  • For example, a change in encoding for a response field from plaintext to base64.
• Semantic changes to request parameters
  • For example, a change from treating the % symbol as a literal instead of as a wildcard or a change in default value.
• Semantic changes to an API endpoint
  • For example, an endpoint that was documented to return all records by default and now returns no records by default.

Non-breaking Changes

• Metadata related changes in behavior
  • For example, when Sugar metadata (Vardefs, Viewdefs, etc.) changes are introduced during a Sugar release or as a result of a customization then REST API responses can vary as a result.
• Adding a new endpoint
  • For example, adding a GET rest/Accounts/:id/map endpoint.
• Adding a parameter to a request
  • For example, adding a boolean parameter called shorten_text that decides if long text fields are shortened to 200 characters.
• Adding a field to a response
  • For example, adding a _self field with link to resource URI in GET responses.
• Adding support for another HTTP request method
  • For example, adding support for PATCH method to an endpoint.

If you ever encounter a situation where there is a breaking change on a particular API version between future Sugar releases, then please report those issues via the Sugar Support portal.

Deprecation process for old API versions

Old REST API versions will continue to be supported until at least one year after they have been deprecated. We want to exercise good judgement on how quickly we remove old API versions but please understand that SugarCRM cannot indefinitely support an unlimited number of REST API versions.

What should you do?

1. When building new Sugar clients or integrations or updating existing ones, adopt the latest available REST API version possible in order to take advantage of all the latest features.
2. Use the HTTP Accept header to specify REST API version when using Sugar Fall ’17 release or later.
3. Monitor the release notes and documentation on each new Sugar release for deprecation notices on old API versions. Keep our customers safe by planning ahead!

Thorn: Test now, REST later

RESTful APIs have always been a big part of the Sugar application, as well as the most appropriate touchpoint for integrating with other enterprise systems. Unfortunately, for a long time they weren’t tested especially appropriately. Traditionally, REST APIs at Sugar were tested with PHPUnit, but only on the classes themselves – they weren’t actually tested via direct use of HTTP. Worse, the tests were messy – they often assumed existing data was present, and when it wasn’t, it was created and not deleted afterwards.

Thorn was envisioned as a solution to this problem. It’s intended to test Sugar’s RESTful API directly, abstract away unhelpful boilerplate, leave your database clean, and let you test like a user.

Thorn was open-sourced earlier this year after a brief internal testing period and is now available for any Sugar developer to use from github.com/sugarcrm/thorn. It is open source and available under the Apache License, version 2.0.

This initial post will focus on a technical and philosophical overview of Thorn. For usage information, see the Thorn website.

Unknown platforms to be restricted in Winter ’18 release

Sugar uses platforms to support the needs of multiple Sugar clients.  The Sugar REST API uses the platform parameter to indicate which platform is being used.  If you’d like a refresher on what the platform parameter is and how to use it, check out this blog post.  

In Sugar 7.9, we added a new Platform extension that we advised developers to start using in the Sugar 7.9 Migration Guide.  The Platform extension allows you to indicate a particular custom platform should be allowed when the disable_unknown_platforms configuration setting is on.

Changes coming in Winter ’18 release

In the Winter ’18 release, we will be preventing REST API access to Sugar from unknown platform types.

Sugar has a configuration setting disable_unknown_platforms that controls whether or not unregistered platforms are allowed to be used when logging in using the REST API. The current default value for disable_unknown_platforms is false. In the Winter ’18 release, we will be changing the default to true, which is how it is already reflected in the documentation.

If your integration uses a custom platform, this custom platform will need to be registered in each Sugar instance or your integration will break!

New tutorial! Intro to the Sugar REST API

Are you ready to build an integration with Sugar but not sure where to start?  You’ve come to the right place!

When you want to access or interact with information stored in Sugar, the REST API is a great place to start.  In this tutorial, you’ll learn how to authenticate to the Sugar REST API.  Then you’ll learn how to perform create, read, update, and delete (aka CRUD) operations.

Watch the video tutorial below or view the text-based tutorial at bit.ly/tutorial_rest.

Upcoming security changes to Sugar

In our most recent set of security releases, we made some changes in Sugar that address input sanitization issues reported by a 3rd party security researcher. Conveniently, these issues can be addressed with the input validation and CSRF form authentication frameworks added in Sugar 7.7.0.0 and 7.7.1.0. Both of these frameworks offer “soft” failure modes that will log warnings into the sugarcrm.log instead of fatal exceptions.

Input Sanitization Soft Failures

CSRF form authentication is strictly enforced by default. But, up until now, the default for the input validation framework has been to use soft failure mode. Choosing to make soft failure mode the default was a pragmatic decision to maximize compatibility for Sugar customizations while developers updated customizations and integrations. However, these recent reported vulnerabilities make it clear that it is time to take the next step to more strictly enforce input sanitization.

SugarCRM plans to strictly enforce input validation in upcoming releases. We will also remove the soft failure mode options at that time which will break customizations or integrations that have not adopted CSRF form authentication or pass that do not pass input validation.

Strict enforcement of Input Validation and CSRF Form Authentication

You should enable strict enforcement of the Input Validation and CSRF Form Authentication checks now for two reasons: (1) to ensure that your Sugar customizations and integrations work properly after upgrading to our winter releases and (2) to create the most secure environment for your current users. The configuration settings in question are the Input Validation ‘validation.soft_fail’ flag as well as the CSRF ‘csrf.soft_fail_form‘ flag.

Sugar Cloud has disabled soft failure modes by default but for Sugar On-Site you can adjust these settings for yourself. Add the following lines to your config_override.php file.

$sugar_config['validation']['soft_fail'] = false;
$sugar_config['csrf']['soft_fail_form'] = false;

Strictly enforced checks

In response to security issues, there are now strictly enforced input validation checks that ignore the validation.soft_fail configuration setting. In particular, we added strict validation to the platform authentication parameter used in our REST API. This can have an impact on platform identifiers using characters that are not part of the POSIX portable filename character set.

Also recall the disable_unknown_platforms configuration setting affects the use of platforms. Custom platforms should be registered using the Platforms extension. This check is planned to be enforced in Sugar On-Demand in the future as well.

What you need to do to prepare customizations

In development instances,

Set validation.soft_fail setting to false.

Set disable_unknown_platforms setting to true.

Then run regression tests on your integrations and customizations to very they still work.

In production instances,

Enable warn log level to collect and analyze any input validation or platform name violation warnings.

Other Resources

Slides from the UnCon 2016 session on Sugar’s input validation framework are also available in the Sugar Community.

More information about best practices for using the Platform parameter can also be found on a previous post on this blog.

 

Using e-mail fields correctly in Sugar

Here is an important message from David Wheeler, a long time Software Engineer and Architect at SugarCRM, about using e-mail fields correctly.

E-mail handling is core to CRM software. Almost everyone we know uses multiple e-mail addresses every single day for both personal or work purposes. So it goes without saying that managing a person’s multiple e-mail addresses correctly is essential in your Sugar customizations and integrations.

History of Sugar E-Mail fields

Several years ago, Sugar changed from using email# named text fields (like email1, email2, etc.) to using an e-mail relationship. This was done to better handle multiple e-mails, multiple relationships, and e-mail attributes like opt in or invalid.

However, use of the email1 field remains particularly persistent. We observe many examples of custom code (and some core code) that still use the old email# fields. This is probably because it is convenient to use the email1 field like a regular text field.

But this is out of date, inaccurate, deprecated, and subject to removal in upcoming Sugar releases.

Below we will describe the proper method for using e-mail fields within Sugar customizations and integrations.

How to filter on related fields using Filter API

This post originally appeared on the SynoLab blog hosted by Synolia, an Elite SugarCRM Partner. Yann Bergès describes how you can use a relate filter with the Filter API. He also explores how Sugar does it as well as identifying a drawback to be considered when using this feature.

We all know the moment when you are roaming through source code to find something for a particular purpose and you come across that feature you didn’t expect but you absolutely want to test. This is how I came into the related link filter feature. What do I mean by related link filter? It is a derivative way to filter related data on a One-to-Many relationship by specifying a link name and a target field:

I want all Contacts filtered on their related Account with « Industry » value « Electronics » (use « one » side of the relationship)
I want all Accounts filtered on their related Contacts with « Title » value « President » (use « many » side of the relationship)

This is an advanced use of Sugar 7 Filter API, if you never used it before, have a look at this documentation for detailed information about how filters work:
– SugarCRM Cookbook – The School of REST – Part 3
– Sugar 7.8 Developer Guide – Architecture – Filters
Examples and tests have been made with a Sugar instance PRO 7.8.0.0

Security changes coming in Sugar 7.8

Important security changes in Sugar 7.8

As we near the release of Sugar 7.8, we wanted to update the Sugar Developer community on a couple of important security changes that are coming in this release.

Using the Sugar REST PHP Client

Sugar REST PHP Client

A new open source library for working with Sugar 7’s powerful REST API has just been published! You can view the library in our GitHub account here: https://github.com/sugarcrm/rest-php-client

Full instructions for installation, usage, current included API Endpoints, and ways to contribute can all be found in the GitHub Wiki on the repository.

Who should use it?

The Sugar REST PHP Client was built initially to make working with Sugar instances easier for some of our internal tools. I wanted to provide a quick, easy, and object oriented, way to access Sugar 7’s REST API that could be used by all PHP developers. Any developer who is developing PHP applications that integrate with Sugar 7 should be interested in using the new library. The simplified architecture takes away the hassle of setting up and managing Curl connections, and allows the developer to focus on what matters most, which is working with the data in their Sugar application.