meta data for this page
  •  

This is an old revision of the document!


Hybris' persistence mechanism

When Hybris was first released in 1997, the first version Enterprise JavaBeans (EJB) was still immature and Hibernate ORM didn't exist. So Hybris developed its own proprietary persistence mechanism which generates Java classes from an xml file. These classes and their relations can easily stored into a relational database. As this is the only persistence mechanism supported by Hybris, every backend developer working with the e-commerce suite must understand how it works and its limitations.

Old Jalo classes

There was an evolution in the Hybris ORM. Until version Hybris 4, you could only generate Jalo classes which are Java persistent classes containing business logic. This led to spaghetti code on those classes. Because of this, Models which are Java persistent POJOs were introduced. Because it is a bad practice to use Jalo classes, I won't explain them here.

Defining persistent classes in items.xml

Note: I will use the source code of the Hybris extension Areco Deployment Scripts Manager to explain how the persistence mechanism works.

Every Hybris extension contains the file resources/<extensionName>-items.xml which is used to define the persistent classes which are used by the extension. Let take a closer look to the file arecoDeploymentScriptsManager-items.xml

<itemtype generate="true"
   code="ScriptExecution"
   jaloclass="org.areco.ecommerce.deploymentscripts.jalo.ScriptExecution"
   autocreate="true">
    <deployment table="arscriptexecution" typecode="32101"/>
    <attributes>
        <attribute qualifier="extensionName" type="java.lang.String">
            <description>Location of the deployment script</description>
            <modifiers optional="false" initial="true" />
            <persistence type="property"/>
        </attribute>
        <attribute qualifier="scriptName" type="java.lang.String">
            <description>Name of the deployment script. It is usually the directory name.</description>
            <modifiers optional="false" initial="true"/>
            <persistence type="property"/>
        </attribute>
        <attribute qualifier="result" type="ScriptExecutionResult">
            <description>Final state of the execution</description>
            <modifiers optional="false" initial="true"/>
            <persistence type="property"/>
        </attribute>
        <attribute qualifier="phase" type="SystemPhase">
            <description>When was the deployment script run.</description>
            <modifiers optional="false" initial="true"/>
            <persistence type="property"/>
        </attribute>
        <attribute qualifier="stacktrace" type="java.lang.String">
            <description>Stacktrace if the script was unsuccessful. It is 4KB long.</description>
            <persistence type="property">
                <columntype database="oracle">
                    <value>text</value>
                </columntype>
                <columntype>
                    <value>varchar(4096)</value>
                </columntype>
            </persistence>
        </attribute>
    </attributes>
</itemtype>
  • Each persistent class is called Type in Hybris and each instance is an Item. The code is just the name of the type and it is usually the name of the Java class
  • Each Hybris type has a jaloclass. You shouldn't use it as it is deprecated but for compatibility reasons, it is required and generated by Hybris
  • For each Hybris type the following Java classes are generated:
    • Model Classes like org.areco.ecommerce.deploymentscripts.model.ScriptExecutionModel: Models are persistent POJOs without any business logic. They are saved in a jar file in the core extension and every extension can use them. They are usually attached to a persistence session.
    • Deprecated Jalo Classes like org.areco.ecommerce.deploymentscripts.jalo.ScriptExecution: They used to contain persistence logic plus business logic. The absence of separation of concerns led to spaghetti code
    • DTO Classes like org.areco.ecommerce.deploymentscripts.dto.ScriptExecutionDTO. These are POJOs used by the extension platformwebservices for the marshalling and unmarshalling of the responses and requests. Unlike the models, they can't contain any business logic and can't be attached to a persistence session. They are rarely used
    • Resource Classes like org.areco.ecommerce.deploymentscripts.resource.ScriptExecutionResource: This is a Java class which was used for marshalling and unmarshalling of collections of responses and requests by the extension platformwebservices
  • Database Table
    • Every Hybris type is a subclass of the class Item which uses the table genericitems
    • Hybris puts the items of the Hybris types by default in the table genericitems. This leads to performance issues when the number of Hybris types increases. Due to this it is good practice to declare a deployment table for every direct subclass of Item
    • if you forgot to add a deployment table, there is no way to change the table where the items are stored without doing a initialization of the database. Adding the deployment table to the items.xml later on, don't have any effect
    • Don't specify a second deployment table in your Hybris types because Hybris don't support it. For example, you declare the Type EnergyProduct as subtype of Product with the deployment table energy_products and you have a Hybris type ProductCarouselCMSComponent with the attribute Product. Then you use Hybris' query language, FlexibleSeach, to look for carousels with products with stock, you will get duplicate carousels because Hybris does two joins for the tables products and energy_products with cmscomponents and aggregates them using union all
    • The subclasses of your custom Hybris items will all reside in the same database table
  • Primary Key and type code
    • The typecode is an internal ID for the Hybris type. It can't be changed once the type was created because the primary keys of the items contain the typecode.
    • The typecode ranges 0-10099, 13200-13299, 24400-24599 and 32700-32799 are reserved for internal Hybris' use and must not be used to avoid conflicts with new Hybris out-of-the-box extensions
    • Every item has the primary key (PK), creation time, modified time, itemtype (read-only) attributes which are mapped to columns in the table. They are managed by Hybris
    • There is no way to generate valid PKs outside the Hybris platform and the PK of an item of one database could be invalid in other database. Hybris has internal counters for PKs which aren't accessible to the developers. If you want to import data into another system, you must use Hybris export language Impex and regenerate the PK during the import. Due to this, Hybris items have a secondary key like uid for CMS components or “code and catalog version” for products
  • You can extend Hybris' Types like in the example with the attribut Product.isDangerousGood. You have to use autocreate=“false” and the original jalo class
  • Database indexes can also be defined after the attribute definition

Recommendations

  • Database Tables: Use the deployment tag to specify the table where the instances of this type or relation are going to be stored. A recommended practice is to do this with every type which is a direct subclass of Item or GenericItem and every relation. See https://wiki.hybris.com/display/release4/Specifying+a+Deployment+for+hybris+Platform+Types
  • If you specify a table for a Hybris Type which already has one, like Product, you will see duplicate row in HMC when ordering by the product attribute (Version 6.1). The sql create by the flexible search query service will join the tables incorrectly.
  • Serializable: Declare a serialVersionUID% field if the item could be used in a Collection. Hybris stores Collection in a blog field in the database after serializing the object. You may have incompatibilities with future changes if you don't set your own serialVersionUID. See:
  • Don't change the package of an Hybris type or prepare yourself for unexpected errors during the update running system with the actual instances of the type.

Attributes

  • They are the fields of your persistent Java classes and are going to be mapped to columns on the deployment table
  • The qualifier is the name of the field and is going to be used in the setter and getters names
  • The type can be an simple Java class which can be stored in a table column like Integer, String, Boolean (Atomic types) or a Hybris Type. In the later case, the PK of the Item is stored on the column
  • It is a good practice to always write a description for the attribute
  • You could use modifiers to tell Hybris if:
    • The field is mandatory (optional=false). The table column will allow nulls, the validation is done by Hybris using Java code
    • The field can only be written once (initial=true)
    • The field must be removed when the parent Item is removed (partof=true)
    • The field is part of the secondary key and must be unique inside the cluster of servers (unique=true). A group of attributes could be unique. The validation is done by Hybris using Java code. As two threads may store two items with the same unique values on the database, you have to use database transactions to be sure that the fields are unique across the cluster of servers
  • There are two used persistence types:
    • Property: The value of the field is stored into the database. The type of the column can be defined for each database engine like with the attribute stacktrace above
    • Dynamic: The setter and getter of the field call a Spring Bean which handles the storage of the value in any persistent medium. For example:
<itemtype code="Product" jaloclass="de.hybris.platform.jalo.product.Product" autocreate="false" generate="true">
    <attributes>
    (...)
    <attribute qualifier="dangerousGood" type="java.lang.Boolean" >
            <persistence type="dynamic" attributeHandler="isDangerousGoodAttributeHandler"/>
            <modifiers write="false" optional="false"/>
        </attribute>
    </attributes>
</itemtype>
@Component("isDangerousGoodAttributeHandler")
public class IsDangerousGoodAttributeHandler implements DynamicAttributeHandler<Boolean, ProductModel> {
  @Override
  public Boolean get(final ProductModel pProduct) {
    // If all the fields are empty, the product is not a dangerous good.
    return !(pProduct.getHazardStatementCodes().isEmpty() //
        && pProduct.getHazardStatementTexts().isEmpty() //
        && pProduct.getEuHazardStatementCodes().isEmpty() //
        && pProduct.getEuHazardStatementTexts().isEmpty() //
        && pProduct.getPrecautionaryStatementCodes().isEmpty() //
        && pProduct.getPrecautionaryStatementTexts().isEmpty() //
        && pProduct.getHazardPictographCodes().isEmpty() //
        && pProduct.getHazardPictographDescriptions().isEmpty() //
        && pProduct.getHazardSignalWords().isEmpty() //
        && pProduct.getBiocideNotes().isEmpty());
  }
 
  @Override
  public void set(final ProductModel pProductModel, final Boolean pBoolean) {
    throw new UnsupportedOperationException("This attribute can't be set.");
  }
}    

This handler makes a calculation based on other fields. Every handler must implement the interface DynamicAttributeHandler<AttributeType, TypeModel>

  • Attributes can be localized and there would be one value for each locale. Internally Hybris uses a map<Locale, AttributeType>:
<attribute qualifier="description" type="localized:java.lang.String">
    <description>Description of the state.</description>
    <persistence type="property"/>
</attribute>

The values of these attributes are saved in a separated table with the sufix lp like arscriptresultlp for the type ScriptExecutionResult.

Recommendations

  • Hybris doesn't remove old attributes when you remove them from the files *-items.xml. If you remove a mandatory attribute, you must set it as optional, remove the setters and getters:
<attribute qualifier="id" type="java.lang.String">
  <description>DEPRECATED: hybris doesn't remove old mandatory attributes, so we keep it.</description>
  <modifiers optional="true" read="false" write="false" />
  <persistence type="property"></persistence>
</attribute>

After an update running system in production, you could remove it. This means that it takes two releases to remove it.
The best alternative is to use the Areco Deployment Script Manager and remove the old attribute using a deployment script.

  • Hybris can't guarantee the uniqueness of a group of attributes which have the modifier unique. As the uniqueness is check on the Java code side, you could end with duplicate rows in the database coming from different threads on one server or from different servers in a cluster. A good practice is to declare a unique index for the Hybris type or use transactions for objects which must be consistent like stock.

Relations between Hybris Types

One to many relations

<relation code="Customer2EmailAddress" generate="true" localized="false" autocreate="true">    <sourceElement qualifier="customer" type="Customer" cardinality="one"/>    <targetElement qualifier="emails" type="EmailAddress" cardinality="many" collectiontype="list" ordered="true"/></relation>

This example creates the column customer in the table where the items EmailAddress are stored and getters and setters on the customers to set and get the list of emails. The sourceElement and targetElement may have modifiers like the attributes. You have to read the source and target elements XML tags crossed to understand where the setters and getters are generated.

Many to many relations

<relation code="InterestGroupVoucherRestrictionToInterestGroupRelation" localized="false" generate="true" autocreate="true">    <deployment table="voucherToInterestGroup" typecode="15189"/>    <sourceElement qualifier="voucherRestrictions" type="InterestGroupRestriction" cardinality="many" />    <targetElement qualifier="interestGroups" type="InterestGroup" cardinality="many" /></relation>

This examples creates the table voucherToInterestGroup with the links between voucher restrictions and the interest groups. And generates the fields InterestGroupRestriction.interestGroups and InterestGroup.voucherRestrictions. A deployment table is required.

Hybris Enumerations

<enumtypes>    <enumtype code="SystemPhase" dynamic="false" >        <value code="INITIALIZATION" />        <value code="UPDATE" />    </enumtype></enumtypes>

This static enumeration is generated as the Java enum org.areco.ecommerce.deploymentscripts.enums.SystemPhase and can be referenced in any attribute of Hybris types. The list of values can only changed during compilation time by changing the items.xml file.
If you want to change the values of an enumeration during runtime, creating values with hmc or importing them with Impex you have to declare an enumeration as dynamic:

<enumtype code="StockLevelStatus" generate="true" autocreate="true" dynamic="true">    <description>Flag for real stock level status</description>    <value code="inStock" />    <value code="outOfStock" /></enumtype>

This creates a Java class which has an static field for each code. This class has the field code and from the point of view of a developer, it is used like a Hybris type and you can import or export the items using Impex. If you need more fields in your custom dynamic enumeration, you must declare a Hybris type instead.

Recommendations

  • Hybris doesn't recommend using collections, use relations instead
    • Hybris may remove support for CollectionTypes directly as the type of an attribute starting with Version 5.0.
    • They can't be searched.
    • They are saved as a list of PK in a blog column in the database. This is slow and you can use SQL or flexible Search to join tables by collection attributes.
    • Cases where you can use collections
      • When your field contains a collection of String or Enumeration values.
      • When your field is dynamic because you aren't persisting the collection in the database.
  • Getter of a Collection: '''The returned Collection should not be modified''' because the changes are going to be lost or you are going to get an Exception (See https://wiki.hybris.com/display/release4/Models#Models-Collections and https://wiki.hybris.com/display/forum/Adding+new+item+to+collection). A new Java collection should be used
  • Relations with enumerations: 1-n don't work and m-n can be defined but you may get an error in some machines when running the update running system. A safe approach is to create an item type which relates the enumeration with the other item.
  • One to Many: Either endpoint cannot be copied easily because the information of the relation is kept in the instance of each class.
  • Many to Many: Can be copied and searched. The relations are kept in a separated table, so they can be searched with FlexibleQuery or SQL.
  • Don't create attributes in relations because they aren't supported by the model layer. If you need a relation with fields, create an association object.

How to persist and load items

In every case you must use the modelService:

@Component("deploymentScript2ExecutionConverter")public class DeploymentScript2ExecutionConverter implements Converter<DeploymentScript, ScriptExecutionModel> {  private static final Logger LOG = Logger.getLogger(DeploymentScript2ExecutionConverter.class);  @Autowired  private ModelService modelService;  /*   * (non-Javadoc)   *    * @see de.hybris.platform.servicelayer.dto.converter.Converter#convert(java.lang.Object)   */  @Override  public ScriptExecutionModel convert(final DeploymentScript source) throws ConversionException {    return this.convert(source, (ScriptExecutionModel) this.modelService.create(ScriptExecutionModel.class)); // Creation of a new item  }  /*   * (non-Javadoc)   *    * @see de.hybris.platform.servicelayer.dto.converter.Converter#convert(java.lang.Object, java.lang.Object)   */  @Override  public ScriptExecutionModel convert(final DeploymentScript source, final ScriptExecutionModel execution)      throws ConversionException {    ServicesUtil.validateParameterNotNullStandardMessage("source", source);    ServicesUtil.validateParameterNotNullStandardMessage("execution", execution);    if (DeploymentScript2ExecutionConverter.LOG.isDebugEnabled()) {      DeploymentScript2ExecutionConverter.LOG.debug("Creating an script execution model from the deployment script " + source);    }    execution.setExtensionName(source.getExtensionName());                                                         // The model is filled    execution.setScriptName(source.getName());    execution.setResult(null); // The caller must set the result before saving the execution.    execution.setPhase(source.getPhase());    return execution;  }}@Servicepublic class ArecoDeploymentScriptsRunner implements DeploymentScriptRunner {  private static final Logger LOG = Logger.getLogger(ArecoDeploymentScriptsRunner.class);  @Autowired  private ModelService modelService;(...)  private void saveAndLogScriptExecution(final UpdatingSystemExtensionContext context,      final ScriptExecutionModel scriptExecution) {    this.modelService.save(scriptExecution);                                                            // The model is stored into the database    context.logScriptExecutionResult(scriptExecution);  }}

To retrieve the model you may use ModelService#get(de.hybris.platform.core.PK) but you usually use Hybris Query Language FlexibleSearch to get items from the database.
You can remove an item using ModelService#remove(java.lang.Object).
You can update the item with the contents of the database using ModelService#refresh. If two thread modify simultaneously two instances of the same model containing different values, the last thread is going to override the changes from the first one. If you are updating critical models like stock levels you must do it inside a database transaction. Hybris don't have any optimistic locking mechanism like Hibernate ORM.

Localization of Hybris types and attributes

The names and descriptions of the Hybris types, enumerations, relations and attributes can be localized using a property file inside resources/localization. For example,

File arecoDeploymentScriptsManager/resources/localization/arecoDeploymentScriptsManager-locales_de.properties(...)type.DeploymentEnvironment.name=Deployment Umgebungentype.DeploymentEnvironment.name.name=Nametype.DeploymentEnvironment.description.name=Beschreibungtype.SystemPhase.name=Phase des Systemstype.SystemPhase.INITIALIZATION.name=Initializationtype.SystemPhase.UPDATE.name=Aktualisierung(...)

This sets the name of a Hybris type, an enumeration and two attributes in German.

General Recommendations

  • Don't use the menu HMC > System > Types to modify the Hybris types on runtime because you changes will be lost after an update running system


Based on Hybris 6.3

<!– var jcomments=new JComments(64, 'com_content','/component/jcomments/'); jcomments.setList('comments-list'); //–>

Add comment

Name (required)

E-mail (required, but will not display)

Website

:D:lol::-);-)8):-|:-*:oops::sad::cry::o:-?:-x:eek::zzz:P:roll::sigh:1000 symbols left

Notify me of follow-up comments

Send

Cancel

<!– function JCommentsInitializeForm() {

      var jcEditor = new JCommentsEditor%%('%%comments-form-comment', true);
      jcEditor.initSmiles%%('/%%components/com_jcomments/images/smilies%%/'%%);
      jcEditor.addSmile%%('%%:D','laugh.gif');
      jcEditor.addSmile%%('%%:lol:','lol.gif');
      jcEditor.addSmile%%('%%:-)','smile.gif');
      jcEditor.addSmile%%('%%;-)','wink.gif');
      jcEditor.addSmile%%('%%8)','cool.gif');
      jcEditor.addSmile%%('%%:-|','normal.gif');
      jcEditor.addSmile%%('%%:-%%*'%%,'whistling.gif');
      jcEditor.addSmile%%('%%:oops:','redface.gif');
      jcEditor.addSmile%%('%%:sad:','sad.gif');
      jcEditor.addSmile%%('%%:cry:','cry.gif');
      jcEditor.addSmile%%('%%:o','surprised.gif');
      jcEditor.addSmile%%('%%:-?','confused.gif');
      jcEditor.addSmile%%('%%:-x','sick.gif');
      jcEditor.addSmile%%('%%:eek:','shocked.gif');
      jcEditor.addSmile%%('%%:zzz','sleeping.gif');
      jcEditor.addSmile%%('%%:P','tongue.gif');
      jcEditor.addSmile%%('%%:roll:','rolleyes.gif');
      jcEditor.addSmile%%('%%:sigh:','unsure.gif');
      jcEditor.addCounter(1000, %%''%%, ' symbols left', 'counter');
      jcomments.setForm(new JCommentsForm%%('%%comments-form', jcEditor));

}

if (window.addEventListener) {window.addEventListener('load',JCommentsInitializeForm,false);} else if (document.addEventListener){document.addEventListener('load',JCommentsInitializeForm,false);} else if (window.attachEvent){window.attachEvent('onload',JCommentsInitializeForm);} else {if (typeof window.onload=='function'){var oldload=window.onload;window.onload=function(){oldload();JCommentsInitializeForm();}} else window.onload=JCommentsInitializeForm;} //–>

JComments

  • You are here:  
  • Hybris' persistence mechanism

Discussion

Johannes Ballmann, 2020/09/17 10:20

It is possible to add a partof modifier to either side of a many to many relation. But what is the meaning of it if one side of a many to many relation has the partof modifier? Would the target element be deleted if the first source element is deleted? the last? Does nothing happen, because partof is ignored in many to many relations?

Maik B., 2020/09/30 11:03

Hybris clearly has a mechanism to provide optimistic locking, so unfortunatly the statement “Hybris don't have any optimistic locking mechanism like Hibernate ORM.” is not true. Every table has a field called hjmpTS which is comparable with the hibernate @Version annotation and is auto incremented on every change. This can go even further, as if you do an update on one site of a relation, hybris automagically takes care of updating the modifiedTs and increases the hjmpTs by one for all other sides of the relation.

When saving hybris checks if the hjmpTs is increased and throws the HJMPException based on that. There are some more detailed insights on that: https://hybrismart.com/2020/03/12/concurrent-update-of-sap-commerce-cloud-item-relations/

Enter your comment. Wiki syntax is allowed: