Skip to content

feat(QTDI-1291): support guess schema as service #1220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ozhelezniak-talend wants to merge 3 commits into
base: master
Choose a base branch
from
+294 −774
Open

feat(QTDI-1291): support guess schema as service #1220

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
568a0b1
feat(QTDI-1291): support guess schema as service
ozhelezniak-talend Sep 22, 2025
f15c17d
feat(QTDI-1291): refactor
ozhelezniak-talend Jun 24, 2026
e8bc2fb
feat(QTDI-1291): tests
ozhelezniak-talend Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions ...ent-api/src/main/java/org/talend/sdk/component/api/exception/DiscoverSchemaException.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@
import javax.json.bind.annotation.JsonbCreator;
import javax.json.bind.annotation.JsonbPropertyOrder;

import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import lombok.ToString;

/**
* This class is dedicated to Studio's guess schema feature.
Expand All @@ -29,7 +32,10 @@
*
* See me TCOMP-2342 for more details.
*/
@Data
@Setter
@Getter
@ToString
@NoArgsConstructor
@EqualsAndHashCode(callSuper = true)
@JsonbPropertyOrder({ "localizedMessage", "message", "stackTrace", "suppressed", "possibleHandleErrorWith" })
public class DiscoverSchemaException extends RuntimeException {
Expand Down Expand Up @@ -58,7 +64,7 @@ public enum HandleErrorWith {
* This won't query for any user input.
* When specifying this option, developer should be sure that no side effect can be generated by connector.
*/
EXECUTE_LIFECYCLE;
EXECUTE_LIFECYCLE
}

private HandleErrorWith possibleHandleErrorWith = HandleErrorWith.EXCEPTION;
Expand All @@ -69,20 +75,19 @@ public DiscoverSchemaException(final ComponentException e) {

public DiscoverSchemaException(final ComponentException e, final HandleErrorWith handling) {
super(e.getOriginalMessage(), e.getCause());
setPossibleHandleErrorWith(handling);
this.possibleHandleErrorWith = handling;
}

public DiscoverSchemaException(final String message, final HandleErrorWith handling) {
super(message);
setPossibleHandleErrorWith(handling);
this.possibleHandleErrorWith = handling;
}

@JsonbCreator
public DiscoverSchemaException(final String message, final StackTraceElement[] stackTrace,
final HandleErrorWith handling) {
super(message);
setStackTrace(stackTrace);
setPossibleHandleErrorWith(handling);
this.possibleHandleErrorWith = handling;
}

}
6 changes: 6 additions & 0 deletions ...r-model/src/main/java/org/talend/sdk/component/server/front/model/error/ErrorPayload.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,11 @@ public class ErrorPayload {

private ErrorDictionary code;

private String subCode;

Comment on lines +31 to +32

@undx undx Jun 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've a model change, we need to specify it in jira issue (pinging @acatoire) but, apparently, it doesn't affect TCOMP-api-test.

private String description;

public ErrorPayload(final ErrorDictionary code, final String description) {
this(code, null, description);
}
}
59 changes: 39 additions & 20 deletions ...ponent-server/src/main/java/org/talend/sdk/component/server/front/ActionResourceImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@
import javax.ws.rs.core.Response;

import org.talend.sdk.component.api.exception.ComponentException;
import org.talend.sdk.component.api.exception.DiscoverSchemaException;
import org.talend.sdk.component.api.exception.DiscoverSchemaException.HandleErrorWith;
import org.talend.sdk.component.runtime.manager.ComponentManager;
import org.talend.sdk.component.runtime.manager.ContainerComponentRegistry;
import org.talend.sdk.component.runtime.manager.ServiceMeta;
Expand Down Expand Up @@ -127,7 +129,7 @@
.collect(toList()));
}

private CompletableFuture<Response> doExecuteLocalAction(final String family, final String type,

Check failure on line 132 in component-server-parent/component-server/src/main/java/org/talend/sdk/component/server/front/ActionResourceImpl.java

View check run for this annotation

sonar-rnd / SonarQube Code Analysis

component-server-parent/component-server/src/main/java/org/talend/sdk/component/server/front/ActionResourceImpl.java#L132 

Refactor this method to reduce its Cognitive Complexity from 29 to the 15 allowed.
final String action, final String lang, final Map<String, String> params) {
return CompletableFuture.supplyAsync(() -> {
if (action == null) {
Expand Down Expand Up @@ -176,16 +178,15 @@
// check org.talend.sdk.component.server.service.ComponentManagerService.readCurrentLocale if you change it
}, Runnable::run).exceptionally(e -> {
final Throwable cause;
if (e.getCause() instanceof ExecutionException) {
cause = e.getCause().getCause();
if (e.getCause() instanceof final ExecutionException exece) {
cause = exece.getCause();
} else {
cause = e.getCause();
}
if (cause instanceof WebApplicationException) {
final WebApplicationException wae = (WebApplicationException) cause;
if (cause instanceof final WebApplicationException wae) {
final Response response = wae.getResponse();
String message = "";
if (wae.getResponse().getEntity() instanceof ErrorPayload) {
if (response.getEntity() instanceof ErrorPayload) {
throw wae; // already logged and setup broken so just rethrow
} else {
try {
Expand All @@ -212,32 +213,50 @@

private Response onError(final Throwable re) {
log.warn(re.getMessage(), re);
if (re.getCause() instanceof WebApplicationException) {
return ((WebApplicationException) re.getCause()).getResponse();
if (re instanceof final WebApplicationException webException) {
return webException.getResponse();
} else if (re.getCause() instanceof final WebApplicationException webException) {
return webException.getResponse();
}

if (re instanceof ComponentException) {
final ComponentException ce = (ComponentException) re;
final String description = "Action execution failed with: " + ofNullable(re.getMessage())
.orElseGet(() -> re instanceof NullPointerException
? "unexpected null"
: "no error message");
if (re instanceof final DiscoverSchemaException eSchema) {
// we send reason to recognize the error on client side
final String subCode = ofNullable(eSchema.getPossibleHandleErrorWith())
.orElse(HandleErrorWith.EXCEPTION)
.toString();
throw new WebApplicationException(Response
.status(ce.getErrorOrigin() == ComponentException.ErrorOrigin.USER ? 400
: ce.getErrorOrigin() == ComponentException.ErrorOrigin.BACKEND ? 456 : 520,
"Unexpected callback error")
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR,
"Action execution failed with: " + ofNullable(re.getMessage())
.orElseGet(() -> re instanceof NullPointerException ? "unexpected null"
: "no error message")))
.status(400, subCode)
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR, subCode, description))
.build());
Comment on lines +226 to +234

@ozhelezniak-talend ozhelezniak-talend Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@undx Do you know why we throw an exception instead just return the Result?

Comment on lines +226 to +234
Comment on lines +226 to +234
} else if (re instanceof final ComponentException eComponent) {
throw new WebApplicationException(Response
.status(evaluateStatusCodeForException(eComponent), "Unexpected callback error")
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR, description))
.build());
Comment on lines +236 to 239
}

throw new WebApplicationException(Response
.status(520, "Unexpected callback error")
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR,
"Action execution failed with: " + ofNullable(re.getMessage())
.orElseGet(() -> re instanceof NullPointerException ? "unexpected null"
: "no error message")))
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR, description))
.build());
Comment on lines 242 to 245
}

private static int evaluateStatusCodeForException(final ComponentException eComponent) {
if (null == eComponent.getErrorOrigin()) {
return 520;
}

return switch (eComponent.getErrorOrigin()) {
case USER -> 400;
case BACKEND -> 456;
default -> 520;
};
}

private Stream<ActionItem> findVirtualActions(final Predicate<String> typeMatcher,
final Predicate<String> componentMatcher, final Locale locale) {
return virtualActions
Expand Down
160 changes: 99 additions & 61 deletions ...onent-server/src/main/java/org/talend/sdk/component/server/service/PropertiesService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.util.Collection;
import java.util.Comparator;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
Expand All @@ -35,6 +36,7 @@
import javax.inject.Inject;
import javax.json.bind.Jsonb;

import org.talend.sdk.component.api.record.Schema;
import org.talend.sdk.component.runtime.internationalization.ParameterBundle;
import org.talend.sdk.component.runtime.manager.ParameterMeta;
import org.talend.sdk.component.runtime.manager.reflect.parameterenricher.ValidationParameterEnricher;
Expand Down Expand Up @@ -69,54 +71,89 @@

private Stream<SimplePropertyDefinition> buildProperties(final List<ParameterMeta> meta, final ClassLoader loader,
final Locale locale, final DefaultValueInspector.Instance rootInstance, final ParameterMeta parent) {
return meta.stream().flatMap(p -> {
final String path = sanitizePropertyName(p.getPath());
final String name = sanitizePropertyName(p.getName());
final String type = p.getType().name();
final boolean isEnum = p.getType() == ParameterMeta.Type.ENUM;
PropertyValidation validation = propertyValidationService.map(p.getMetadata());
if (isEnum) {
if (validation == null) {
validation = new PropertyValidation();
}
validation.setEnumValues(p.getProposals());
}
final Map<String, String> sanitizedMetadata = ofNullable(p.getMetadata())
.map(m -> m
.entrySet()
.stream()
.filter(e -> !e.getKey().startsWith(ValidationParameterEnricher.META_PREFIX))
.collect(toLinkedMap(e -> e.getKey().replace("tcomp::", ""), Map.Entry::getValue)))
.orElse(null);
final Map<String, String> metadata;
if (parent != null) {
metadata = sanitizedMetadata;
} else {
metadata = ofNullable(sanitizedMetadata).orElseGet(HashMap::new);
metadata.put("definition::parameter::index", String.valueOf(meta.indexOf(p)));
}
final DefaultValueInspector.Instance instance = defaultValueInspector
.createDemoInstance(
ofNullable(rootInstance).map(DefaultValueInspector.Instance::getValue).orElse(null), p);
final ParameterBundle bundle = p.findBundle(loader, locale);
final ParameterBundle parentBundle = parent == null ? null : parent.findBundle(loader, locale);
return Stream
.concat(Stream
.of(new SimplePropertyDefinition(path, name,
bundle.displayName(parentBundle).orElse(p.getName()), type, toDefault(instance, p),
validation, rewriteMetadataForLocale(metadata, parentBundle, bundle),
bundle.placeholder(parentBundle).orElse(p.getName()),
!isEnum ? null
: p
.getProposals()
.stream()
.collect(toLinkedMap(identity(),
key -> bundle
.enumDisplayName(parentBundle, key)
.orElse(key))))),
buildProperties(p.getNestedParameters(), loader, locale, instance, p));
}).sorted(Comparator.comparing(SimplePropertyDefinition::getPath)); // important cause it is the way you want to
// see it
return meta.stream()
.flatMap(p -> buildProperty(p, meta, loader, locale, rootInstance, parent))
// important cause it is the way you want to see it
.sorted(Comparator.comparing(SimplePropertyDefinition::getPath));
}

private Stream<SimplePropertyDefinition> buildProperty(final ParameterMeta p,
final List<ParameterMeta> siblings,
final ClassLoader loader,
final Locale locale,
final DefaultValueInspector.Instance rootInstance,
final ParameterMeta parent) {
final String path = sanitizePropertyName(p.getPath());
final String name = sanitizePropertyName(p.getName());
final String type = p.getType().name();

final PropertyValidation validation = buildValidation(p);
final Map<String, String> metadata = buildMetadata(p, siblings, parent);

final DefaultValueInspector.Instance instance = defaultValueInspector
.createDemoInstance(ofNullable(rootInstance)
.map(DefaultValueInspector.Instance::getValue)
.orElse(null), p);
final ParameterBundle bundle = p.findBundle(loader, locale);
final ParameterBundle parentBundle = parent == null ? null : parent.findBundle(loader, locale);
final String displayName = bundle.displayName(parentBundle).orElse(p.getName());
final String placeholder = bundle.placeholder(parentBundle).orElse(p.getName());

final LinkedHashMap<String, String> enumValues = buildEnumDisplayNames(p, bundle, parentBundle);
final SimplePropertyDefinition def = new SimplePropertyDefinition(path, name, displayName, type,
toDefault(instance, p), validation, rewriteMetadataForLocale(metadata, parentBundle, bundle),
placeholder, enumValues);

return Stream.concat(
Stream.of(def),
buildProperties(p.getNestedParameters(), loader, locale, instance, p));
}

private PropertyValidation buildValidation(final ParameterMeta p) {
PropertyValidation validation = propertyValidationService.map(p.getMetadata());
if (p.getType() != ParameterMeta.Type.ENUM) {
return validation;
}
if (validation == null) {
validation = new PropertyValidation();
}
validation.setEnumValues(p.getProposals());
return validation;
}

private Map<String, String> buildMetadata(
final ParameterMeta p,
final List<ParameterMeta> siblings,
final ParameterMeta parent) {
final Map<String, String> sanitized = ofNullable(p.getMetadata())
.map(m -> m
.entrySet()
.stream()
.filter(e -> !e.getKey().startsWith(ValidationParameterEnricher.META_PREFIX))
.collect(toLinkedMap(e -> e.getKey().replace("tcomp::", ""), Map.Entry::getValue)))
.orElse(null);
if (parent != null) {
return sanitized;
}

final Map<String, String> metadata = ofNullable(sanitized).orElseGet(HashMap::new);
metadata.put("definition::parameter::index", String.valueOf(siblings.indexOf(p)));
// this one to mark the Schema parameter somehow to differentiate it from the branch name
if (p.getJavaType() instanceof Class<?> clazzType && Schema.class.isAssignableFrom(clazzType)) {
metadata.put("definition::parameter::schema", "");
}
return metadata;
Comment on lines +139 to +145
}

private LinkedHashMap<String, String> buildEnumDisplayNames(final ParameterMeta p, final ParameterBundle bundle,
final ParameterBundle parentBundle) {
if (p.getType() != ParameterMeta.Type.ENUM) {
return null;

Check warning on line 151 in component-server-parent/component-server/src/main/java/org/talend/sdk/component/server/service/PropertiesService.java

View check run for this annotation

sonar-rnd / SonarQube Code Analysis

component-server-parent/component-server/src/main/java/org/talend/sdk/component/server/service/PropertiesService.java#L151 

Return an empty map instead of null.
}

return p.getProposals()
.stream()
.collect(toLinkedMap(identity(), key -> bundle.enumDisplayName(parentBundle, key).orElse(key)));
}

private Map<String, String> rewriteMetadataForLocale(final Map<String, String> metadata,
Expand Down Expand Up @@ -151,20 +188,21 @@
return metadata;
}
final Predicate<Map.Entry<String, ?>> shouldBeRewritten = k -> keysToRewrite.contains(k.getKey());
return Stream
.concat(metadata.entrySet().stream().filter(shouldBeRewritten.negate()),
metadata
.entrySet()
.stream()
.filter(shouldBeRewritten)
.map(it -> new AbstractMap.SimpleEntry<>(bundle
.gridLayoutName(parentBundle,
it
.getKey()
.substring("ui::gridlayout::".length(),
it.getKey().length() - "::value".length()))
return Stream.concat(
metadata.entrySet()
.stream()
.filter(shouldBeRewritten.negate()),
metadata.entrySet()
.stream()
.filter(shouldBeRewritten)
.map(it -> new AbstractMap.SimpleEntry<>(
bundle.gridLayoutName(parentBundle,
it.getKey()
.substring("ui::gridlayout::".length(),
it.getKey().length() - "::value".length()))
.map(t -> "ui::gridlayout::" + t + "::value")
.orElse(it.getKey()), it.getValue())))
.orElse(it.getKey()),
it.getValue())))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}

Expand Down
Loading
Loading